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Report of the Auditor General 

to the House of Commons 

for December 1998 


Foreword 


I am pleased to present the third volume of my 1998 Report. This Foreword is followed by “Matters of 
Special Importance — 1998” and the Main Points from all of this year’s chapters. In addition, this volume contains 
11 chapters, bound separately: 


19. Electronic Commerce: Conducting Government Business via the Internet 
20. Preparedness for Year 2000: Government-Wide Mission-Critical Systems 
UIE Canadian International Development Agency — Geographic Programs 


Os. The Federal Science and Technology Strategy: A Review of Progress 


23: Veterans Affairs Canada — Disability Pensions 

24. Revenue Canada — International Tax Directorate: Human Resource Management 

25, Transport Canada — Investments in Highways 

26. Contracting for Professional Services: Selected Sole-Source Contracts 

8 Grants and Contributions: Selected Programs in Industry Canada and Department of 


Canadian Heritage 
28. Follow-up of Recommendations in Previous Reports 
oo: Other Audit Observations 


My Office issued the first volume of the 1998 Report in April, containing nine chapters, and the second 
volume with nine chapters in September. 


In addition, this year my Office has provided: 
e an opinion and observations on the Financial Statements of the Government of Canada 


e an auditor’s report and observations on the Debt Servicing and Reduction Account Statement of 
Transactions 


e auditor’s reports and observations on some 90 Crown corporations and other entities, territorial 
governments and organizations, and international organizations. 


In 1997-98 my Office completed special examinations of the Farm Credit Corporation, National Capital 
Commission, Great Lakes Pilotage Authority, Cape Breton Development Corporation and Atlantic Pilotage 
Authority. Further, in the subsequent period April to October 1998, my Office also completed special 
examinations of Atomic Energy of Canada Limited, Defence Construction (1951) Limited, National Arts Centre 
Corporation and Via Rail Canada Inc. 


Under section 11 of the Auditor General Act, I may from time to time undertake assignments at the 
request of the Governor in Council. In June I reported to the Minister of National Revenue and the Minister of 
Finance on one such examination of the requirement to report specified foreign property under section 233.3 of 
the Income Tax Act. 
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Matters of Special Importance — 
1998 


Main Points 


Each year in this chapter I highlight significant issues arising from our work of the year, and 
matters from previous years that continue to be of special importance. This year, with the era of large, 
chronic deficits seemingly behind us, I discuss the challenge of remaining focussed on improving 
economy and efficiency in government. 


I suggest a number of means to help ensure that the turnaround in our immediate fiscal situation 
does not lull us into complacency or deflect attention from the need to improve performance: 


e A long-term fiscal framework. For some time now, I have called on the federal government 
to make public long-term fiscal projections showing the effects on government finances of 
factors such as population aging. Governments in other countries are doing this. 


e Effective governance structures. The federal government, like governments elsewhere, is 
trying to move from a traditional bureaucratic model to a more flexible form of management. 
This requires good governance, with attention to serving the public interest, achieving 
objectives, ensuring accountability and maintaining transparency. 


e Innovation and sensible risk taking in the public sector. There are constraints to 
innovation and sensible risk-taking in the public sector. We need to identify these constraints 
more clearly, along with the actions that can be taken to encourage public servants to 
discharge their responsibilities with creativity, innovation and integrity. 


e Modernized financial management and reporting. Modernization of financial 
management is long overdue. As well, the government’s commitment to objective accounting 
standards needs to be reaffirmed — for two years now, I have felt obliged to qualify my audit 
opinion on the government’s financial statements because they fail to meet accepted 
accounting standards. 


e Better integration of people management with ongoing public sector reforms. There is 
much dissatisfaction among government officials with the current state of human resource 
management in the public service. It is troubling that these long-standing problems still exist. 


e Open and meaningful accountability to Parliament. Significant steps are being taken to 
provide parliamentarians with the kind of timely, relevant information they need to hold 
government accountable. The move to alternative service delivery arrangements requires 
attention to prevent a weakening of accountability and cohesion in the public sector. 
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Main Points (cont’d) 


In this post-deficit environment, these elements of good public sector management can help to 
ensure that the pursuit of better value for the taxpayer’s money continues. 


Other recent and ongoing matters are discussed in a supplement to the chapter: 
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Our first assessment of performance information by a new service delivery agency was 
completed this year. 


Our examination of Revenue Canada’s efforts to promote integrity among its employees 
continues our ongoing work on ethical conduct in the public sector. 


We examined the government’s requirement that specified foreign property be reported for 
tax purposes. 


Progress in implementing the government’s science and technology strategy has been slow in 
some important areas. 


Delays in fixing Year 2000 computer problems mean that many government-wide critical 
functions remain at risk. 


Moving beyond the rhetoric of sustainable development to apply its principles in the 
day-to-day operations of government remains a key challenge. 


Over the past year, our audits have identified opportunities for cost savings. 


Vision 
We are committed to making a difference for the Canadian people by 


promoting, in all our work for Parliament, answerable, honest and productive 


government that reflects a commitment to sustainable development. 


Mission 


The Office of the Auditor General of Canada conducts independent audits 
and examinations that provide objective information, advice and assurance 
to Parliament. We promote accountability and best practices in 


government operations. 


Elaboration of Mission 


In achieving our mission, 


we want to make a difference by promoting: 


® a fair and frank accounting of government’s stewardship of financial and 


other resources; 


@ 


efficiency and productivity in the public service; 


5 


cost effectiveness of government activities; 


® collection of revenues owed to the Crown. 


Other effects we want to produce 


through our work are: 


® objective assurance on matters found to be satisfactory and unsatisfactory; 


® 


compliance with authority; 


® deterrence of fraud and dishonesty. 
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Public Management in a Post-Deficit Environment 


he prospect of being hanged, 

Samuel Johnson observed, 

concentrates the mind wonderfully. 
In the early 1990s, Canada’s public finances — 
already weak from years of continuous deficits 
and mounting debts — were brought to the 
brink of crisis by an unusually long and deep 
recession. In 1993-94, the annual budget 
deficit reached a record $42 billion and the net 
debt as a share of GDP rose to over 70 percent, 


compared with less than 20 percent two 
decades earlier. The spectre of financial crisis encouraged the government to 
rethink radically what it does and how it does it. 


Significant changes in federal public administration followed. A new 
Expenditure Management System was introduced, establishing multi-year 
funding targets and requiring that new spending initiatives be financed 
through reallocation within those targets. Programs were privatized; others 
were restructured and streamlined. The size of the public service work force 
was reduced in a meaningful way for the first time in decades, as was 
program spending. Emphasis was placed on management becoming more 
results-oriented and cost-conscious. 


: 4 n'y ; With the era of large chronic deficits seemingly behind 
With large chronic deficits seemingly us, the challenge now is to maintain the momentum 


behind us, the challenge now is to toward greater economy and efficiency in managing 


hala public funds. Eliminating the deficit is only a way station 
malaise nome mun OWwald to restored fiscal health — not the end of the road. Our 


greater economy and efficiency. debt and tax burdens remain high, and current 
~~. demographic trends suggest additional fiscal pressures 
ahead. The global economic upheavals of recent months provide a vivid 
reminder of how quickly economic fortunes can change. It is thus important 
that we not allow the turnaround in our immediate fiscal situation to lull us 
into complacency or to deflect our efforts to improve performance. 


In this chapter, I discuss some means that could support those efforts. They 


include: 
e along-term fiscal framework; 


e effective governance structures; 
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e innovation and sensible risk-taking in the public sector; 
e modernized financial management and reporting; 


e better integration of people management with ongoing public sector 
reforms; and 


e open and meaningful accountability to Parliament. 


Together, these elements would constitute a public management culture that 
in my view would stand us in good stead generally, and particularly in the 
current post-deficit environment. 


Other issues that are not directly related to the theme of managing in a 
post-deficit environment but that I consider important are reported in a 
supplement to this chapter. 


A Long-Term Fiscal Framework 


Jacob’s son Joseph saved Egypt from famine — and made himself rich and 
famous in the process — by counselling Pharaoh to look beyond the 
immediate years of plenty, and to budget accordingly. In making budget 
decisions, a forward view is nothing more than simple prudence. It is 
especially wise when we have reason to expect, as Joseph did, that our 
circumstances in the long term may be materially different from our present. 


We lack Joseph’s ability to foretell the future through the interpretation of 
dreams, but it is easily within our power to extrapolate the long-term fiscal 
implications of existing demographic trends. Demographic developments 
take decades to unfold, and follow a predictable pattern as they do: today’s 
youth will be tomorrow’s workers in a decade or two and Canada’s retirees 
three to four decades after that. In a real sense therefore, as management 
expert Peter Drucker has put it, demographics represent “a future that has 
already happened.” 


What does that future look like? We know that owing to lower fertility and 
mortality rates, Canada’s population is aging. The proportion of elderly in the 
total population will grow sharply during the second and third decades of the 
next century, as baby boomers age and leave the labour force for retirement. 
By 2030, Canada’s elderly will represent 22 percent of the total population, 
compared with 12 percent today. Whereas today there are five Canadians of 
working age for every person 65 years or older, in three decades there will be 
only half that number. 


Given current policies, this demographic shift will impose huge fiscal 
pressures on government, as demands on our pension and health care 
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programs increase while the proportion of the working age population 
supporting these programs shrinks. A fiscal outlook limited to a two-year 
horizon is clearly far too short to take these pressures into account. That 
focus was effective in a period of crisis, but now it is time to look beyond the 


short run. 


Among developed countries at least, there is a growing recognition of the 
advantages of long-term fiscal planning. Recent reports of the Organization 
for Economic Co-operation and Development (OECD) have stressed the 


Budgeting in New Zealand 


Budget making and reporting in New Zealand are 

governed by the Fiscal Responsibility Act 1994. 

The Act requires that fiscal policy be made in 

accordance with the following principles of 

responsible fiscal management: 

e Reducing government debt to prudent levels 

e Matching operating expenses with revenues over 
time 

e Pursuing policies consistent with stable and 
predictable tax rates over time 


These principles find concrete expression in two 
publications: a budget policy statement, to be tabled 
at least three months before budget time, and a fiscal 
strategy report to be tabled with the budget. 


The policy statement sets out the government’s 
fiscal intentions for the ensuing three years and its 
long-term fiscal objectives. 


The strategy report compares the budget objectives 
with those in the policy statement and provides 
“progress outlooks” that project the government's 
fiscal position for 10 years or more, under a range 
of economic and policy assumptions. 


The Fiscal Responsibility Act has been the model for 
similar legislation in Australia and the UK. 


value of long-term budget frameworks in forcing people 
to think about the future. New Zealand, Australia and the 
United Kingdom, countries with parliamentary systems of 
government similar to our own, have adopted legislation 
that obliges their governments to provide long-term fiscal 
outlooks as part of their budget processes. In the U.S., 
both the Congressional Budget Office, on behalf of 
Congress, and the Office of Management and Budget, on 
behalf of the Administration, prepare and make public 
comprehensive projections of revenues, expenditures and 
budget balances under various fiscal and economic 
assumptions for several decades ahead. 


I have at various times called for similar projections for 
Canada. The government’s reluctance to accede to these 
calls stems largely from an expressed concern that it will 
be held accountable for the projected long-term outlooks, 
in the same way it is now accountable for its short-term 
fiscal targets. One way around that fear is to make it clear, 
either through a policy statement accompanying the 
projections or through legislative enactment if necessary, 
that the projections are provided for information purposes 
only and do not represent government commitments — 
which would be meaningless, anyway, given the normal 


length of a typical government’s mandate. In addition, long-term projections 


need not contain the same precision and type of information now contained 


in the government’s two-year forecasts. 


Presentation of these long-term projections as part of the fiscal update in the 
fall, rather than with the annual Budget in February, would also help distance 
them from the government’s short-term fiscal plan and targets. At the same 
time, it would provide useful context for the pre-Budget consultations that 


are the reason for producing the fall fiscal update. 
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I was pleased recently to see the Standing Committee on Public Accounts 
endorse the need for a long-term outlook in budget making. In its 16th report 
tabled last October, based on hearings on Chapter 6 of my 1998 Report, the 
Committee calls on the Department of Finance to devise appropriate means 
for the government to publish long-term demographic and fiscal information 
that would ensure transparency and greater understanding of Canada’s fiscal 
prospects. I hope the Department will respond favourably to the Committee’s 
call. 


Long-term fiscal projections are necessarily speculative, but necessary 
nevertheless: one cannot ignore the future just because it is not perfectly 
predictable. Current fiscal decisions have long-term consequences, whether 
we recognize them or not. Demographic trends are as reliable as any, and 
their fiscal implications can easily be taken into account. Doing so and 
making the results of the exercise public would help to make the 
government’s short-term targets more intelligible — and hence more 
acceptable to the public and more credible to financial markets. 


For related information, see Auditor General publications: Population Aging and 
Information for Parliament: Understanding the Choices (1998 Report, Chapter 6); 
Information for Parliament — Deficits and Debt: Understanding the Choices 
(1995 Report, Chapter 9); Toward Better Governance — Public Service Reform in 
New Zealand 1984-94 and its Relevance to Canada (1995); Information for 
Parliament — Understanding Deficits and Debt (1993 Report, Chapter 5). Other 
publications: Maintaining Prosperity in an Ageing Society (OECD, 1998); 
Long-Term Budgetary Pressures and Policy Options (Congressional Budget Office, 
Washington, DC, May 1998). 


Effective Governance Structures 


Ideas about what works in the public sector, and what effective public 
management is, have undergone a sea change over the past decade or so. 
Traditional public management is characterized by hierarchical 
organizational structures, centralized decision-making, and regulation of 
operating departments through controls imposed by central agencies. It is 
based on detailed rules, rather than managerial judgment; on process, not 
results. 


These bureaucratic features of traditional public management were designed 
to ensure equity across government activities and probity in the stewardship 
of public resources. But they can also make the public sector unwieldy, 
inflexible and inefficient. In a world of rapid change, mounting competition 
and a more demanding public, this traditional bureaucratic model of public 
management is becoming less and less acceptable. 
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In country after country within the industrialized world, this old-style 
approach to public sector management is giving way to a more decentralized, 
flexible management, geared to results. and performance. Policy direction and 
overall budget allocations still emanate from the centre, but operating 
departments have greater autonomy in the way they use 


In country after country in the resources to meet client needs. Accountability is achieved 
Pddeiiaiizedaocddtha on style through the establishment of appropriate performance 
; = 
approach to public sector 
management is giving way to a more Through a number of initiatives under the general rubric 
decentralized. flexible management of “getting government right’, the federal government has 
4 s 


objectives and reporting requirements. 


embarked on a road toward results-based management in 
the public sector. Critical to the success of this approach 
are control frameworks and governance procedures that foster commitment 
to organizational objectives and “make” managers manage. Four key 
principles related to a framework for effective governance are suggested for 
organizations in the public sector: 


e Serving the public interest. Public organizations must balance 
management precepts, such as cost effectiveness, with public objectives 
as stated in legislation and reported to Parliament. An organizational 
climate that both supports public service values and encourages 
managing for results can contribute to performance by guiding behavior 
and motivating effort. 


e Achieving objectives. The organization needs well-defined outcome 
objectives, clear performance expectations and reliable indicators to 
measure progress. It requires the capabilities appropriate to its mission 
and objectives, including the necessary authorities, people and financial 
resources. 


e Ensuring accountability. Clear objectives are key to holding 
management accountable for the public resources allocated to them. 
Roles and responsibilities need to be well understood and agreed upon. 
Performance expectations should be linked to and balanced with 
capabilities. Reports of what an organization has accomplished should be 
timely and reliable, to facilitate learning and promote better performance. 


e Appropriate transparency. Public reporting and public availability of 
key documents are necessary if organizations are to demonstrate their 
stewardship of public funds and fulfilment of the responsibilities 


assigned to them. 
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No management process can guarantee success, of course. The best it can do 


is to keep an organization focussed on the goals it wants to achieve and deal 


Contracting Activities — a Poor Example for 
Managerial Discretion 


Contracting with outsiders for the purchase of goods 
and services is one area where government 
managers enjoy considerable discretion. 
Unfortunately, as currently practised, managerial 
discretion in contracting is often a poor example for 
emulation in other areas. 


In this year’s audit of sole-sourced contracts for 

professional services, we found: 

e Widespread non-compliance with contracting 
regulations 

e Poor documentation 

e Insufficient regard to economy 


To succeed, devolution of authority must be coupled 
with clear performance expectations and reporting 
requirements. Otherwise, we risk eroding both 
accountability and performance. 


See Contracting for Professional Services: Selected 
Sole-Source Contracts (1998 Report, Chapter 26). 


with things under management’s control that get in the 
way of success. An effective governance framework is 
designed to help management do that. 


“Easier said than done,” I can hear some readers say. I 
agree. In particular, I do not underestimate how difficult it 
is in many parts of government to establish clear 
objectives and performance measures, and to report 
information in an open and frank manner. Nevertheless, as 
practices in other countries show, we can do more in this 
area than we are now doing. The potential benefits are 
worth the effort, especially at this time: structures that 
promote a performance ethic, reinforced by disclosure 
practices that facilitate oversight, can minimize the risk of 
backsliding as short-term fiscal pressures to economize 
recede. 


In the remaining sections of this chapter, I examine 
different aspects of this challenge. 


For related information, see Auditor General publications: Reporting Performance 
in the Expenditure Management System (1997 Report, Chapter 5); Moving toward 
Managing for Results (1997 Report, Chapter 11); Crown Corporations: Making 
Performance Measurement Work (1997 Report, Chapter 22); Crown Corporations: 
Fulfilling Responsibilities for Governance (1995 Report, Chapter 10). 


Innovation and Sensible Risk Taking in the Public Sector 


Other sections of this chapter discuss the profound changes — in governance 


structures, in resources, in service delivery methods — that have 


characterized the federal public service in recent years. While many of these 


changes stemmed from the government’s fight against chronic deficits, other 


factors such as globalization, demographic and societal trends, and the 


technology revolution have made rapid change a fact of life. The competitive 


and results-oriented environment that now prevails places a premium on the 


ability and willingness of public servants to take sensible risks and to be 


innovative — not only to keep up with change, but to lead and influence its 


direction, and to seize the opportunities that change presents. 


There are forces, however, that many observers believe militate against risk 


taking and innovation. These include the effects on public servants’ morale 


of such interrelated factors as downsizing, the long wage freeze that ended 
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only last year, and a perceived lack of support. At the same time, more 
openness and a greater level of public accountability for results can 
sometimes unleash undeserved criticism when results fall short of 
expectations. In these circumstances, many public servants may feel the most 
prudent course is to play it safe. 


I recognize that the work of this Office is seen by some as contributing to the 
forces that inhibit innovation. And given the very nature of our mandate, 
which requires the public reporting of instances where things have gone 
wrong, there will always be a possibility for our work to have a stifling effect 
on the willingness of some public servants to innovate and to take risks. 
Nevertheless, I am convinced that this Office is a relatively small player in 
the complex operational environment of the public service. We — the public, 
parliamentarians, public servants and this Office — 


We will continue to work with have to work together to help remove the constraints 
and find ways to encourage the innovation and 


departments and agencies to identify responsible risk taking that are required now more than 
more clearly the constraints on ever. 


innovation and sensible risk taking. eine ; 
It was in the light of these considerations that in 


October this year I sponsored a roundtable discussion 
on innovation and risk taking among senior managers from the federal and 
provincial governments, political representatives, and representatives from 
relevant private sector and labour organizations.* The discussions were 
based on, and informed by, background papers prepared by the Public Policy 
Forum (The Risk Not Taken), Industry Canada (An Industry Canada 
Perspective) and this Office (The Role and Perspective of the Office of the 
Auditor General of Canada). I believe the attendance at the roundtable, the 
lively discussion, and the range of views expressed reflect the importance of 
the topic as well as the interest and concern of the participants. I am 
particularly encouraged by the fact that participants were able to go beyond 
identifying problems toward suggesting some concrete steps that may help 
deal with these problems. 


I see the roundtable as the beginning, rather than the end, of an ongoing 
process. We will continue to work with departments and agencies to identify 
more clearly the constraints on innovation and sensible risk taking and the 
actions that can be taken to encourage public servants to discharge their 
responsibilities with creativity, innovation and integrity. 


* A summary of the proceedings of that roundtable is available by contacting this Office. 
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Modernized Financial Management and Reporting 


Encouraging better financial management in the federal government has been 


one of my priorities as Auditor General since my appointment in 1991. An 


integral component of good financial management is good financial 


information. Indeed, accurate and timely financial information is essential to 


all aspects of public management, from resource allocation and planning to 


monitoring compliance and evaluating performance. Timely and credible 


reporting of relevant financial information is also fundamental to holding 


government departments, and the government as a whole, accountable for the 


resources they use and the power they wield. 


As I look at the state of financial management, I have three key concerns. 


Concern 1: Modernization of financial management is long 


overdue 


Modernizing Comptrollership — Time for 
Action 


Improvements in the government’s financial 
management system are taking place alongside a 
broader initiative to renew comptrollership in 
government. In its 1997 report, the Independent 
Review Panel on Modernization of Comptrollership 
in the Government of Canada argued that the 
comptrollership function — the way it is now 
viewed and practised — must change fundamentally 
to be appropriate to the needs of modern 
government. 


The Panel put forth the following indicators for 
ascertaining whether modernization of 
comptrollership has taken place: 

e The availability of government-wide 
performance standards, appropriately 
implemented to the circumstances of the 
separate departments 

e The provision of comprehensive and credible 
performance information to decision makers 

e The existence of centres of excellence for the 
training of managers and professionals in the 
requirements of comptrollership. 


The government has accepted the recommendations 
of the Panel. I support the Comptroller General’s 
efforts to implement them expeditiously. 


The need to reform the way the government manages 
financial information has been on the public agenda for 
many years. Nearly four decades ago, the Royal 
Commission on Government Organization (Glassco 
Commission) called for the adoption of an accounting 
system that would provide better information on the full 
costs of government activities. The call has been repeated 
many times since then from many quarters, both within 
and outside government. 


The government is now in the process of introducing such 
a system, under an initiative called the Financial 
Information Strategy (FIS). Among other things, FIS will 
replace the existing partly cash-based system of accounts 
with full accrual accounting. This means that spending on 
capital assets will be recognized as the assets are used up, 
so that costs will be distributed over the useful life of 
these assets. Currently, the total cost of a capital asset is 
charged to expenditures at the time the asset is acquired. 
By appropriately associating costs with the operations 
they support, accrual-based accounting will provide us 
with better information on what government programs 
actually cost. 


Modernization of the government’s financial management is long overdue. 
FIS itself was originally approved by Treasury Board in 1989 and was 
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re-endorsed as a government priority in 1995. But implementation has been 
slow. 


Although progress has accelerated recently, we are still years 
away from having a financial management system that 


financial management system that would meet acceptable standards in the private sector. 
would meet acceptable standards in Further delay is not an option: the government needs 


the private sector. 


financial information that it does not now have. 


Concern 2: Government plans would create 
inconsistencies between parliamentary appropriations and 
government expenditures 


As we reported in September, a key challenge of FIS will be to persuade 
departmental officials to use the new financial information in their 
day-to-day management of programs and activities. This is more likely to 
occur if funds are also appropriated on a full accrual basis. Announced 
government plans entail full accrual accounting for the annual Budget, the 
Public Accounts of Canada, departmental Reports on Plans and Priorities and 
departmental Performance Reports, but not for the Estimates submitted to 
Parliament for appropriation purposes. 


In my view, this fissure in the structure of financial reporting could seriously 
impair the effectiveness of FIS. Commitment to accrual principles of 
accounting and budgeting would be lessened among program managers if 
those principles were not used for so basic a purpose as the granting of 
supply. It would also needlessly complicate the accountability regime by 
maintaining essentially two sets of books for the same activities, one for 
managing and one for reporting. Full benefits from FIS will not be realized 
unless it encompasses the supply process, so that the information used by 
public sector managers to plan and run programs is the same as that used by 
Parliament for purposes of resource allocation and approval. 


Concern 3: The government’s commitment to objective accounting 
standards needs to be reaffirmed 


For me, one of the most encouraging aspects of FIS is the stated intention to 
follow accounting principles that accord with the recommendations of the 
Public Sector Accounting and Auditing Board (PSAAB) of the Canadian 
Institute of Chartered Accountants. These standards are developed openly, 
with full participation from the financial, accounting and public sector 
communities, and after due process that involves circulating exposure drafts 
and evaluating responses from interested parties. 
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Compliance with accounting 


Yet for three years running now, I have expressed concerns about the 
government’s failure to comply with PSAAB standards in recording certain 
expenditures in the Public Accounts. Because of this, I have been obliged to 
qualify my opinion on the government’s annual financial statements in each 
of the past two years. The expenditures in question included transitional 
assistance in 1996 for harmonizing provincial sales taxes with the GST, and 
funds to establish the Canada Foundation for Innovation in 1997 and the 
Canada Millennium Scholarship Foundation earlier this year. 


Compliance with accounting standards recommended by 
an independent authority, like the PSAAB, should be the 


standards recommended by an general practice in government accounting. Accounts so 
independent authority, like the Public prepared are likely to represent the consensus of 


Sector Accounting and Auditing 


financial, accounting and public sector experts on how 
best to reflect financial reality in government financial 


Board, should be the general practice statements. In addition, because they will not be tainted 
in government accounting. by suspicion that the method followed was chosen for 


political or partisan advantage, the resulting reports will 
have greater credibility both among citizens and in the national and 
international financial communities. 


The recent Asian financial crisis attests to the importance of transparent, fair 
and complete information on the financial condition of governments. The 
International Monetary Fund identified lack of transparency as a contributing 
factor to that crisis. Opaque reporting practices and limited availability of 
data obstructed a realistic view of economic fundamentals in the countries 
now embroiled in economic troubles. Understandable, timely reporting of 
useful information contributes to better performance, by supporting informed 
decision-making and exposing the activities of government to the discipline 
of public scrutiny. 


For related information, see Auditor General publications: The Financial 
Information Strategy: A Key Ingredient in Getting Government Right (1998 Report, 
Chapter 18); Reporting Government Financial Results: The Importance of 
Complying with Objective Accounting Standards (1998 Report, Chapter 9); 
Financial Management: Developing a Capability Model (1997 Report, Chapter 2); 
Management of the Government's Accounting Function: A Central Agency 
Perspective (1997 Report, Chapter 3). Other publication: Observations by the 
Auditor General (Public Accounts of Canada, Volume 1, Section 1, 1996 to 1998). 


Better Integration of People Management with Ongoing 
Public Sector Reforms 


One of the biggest challenges of the post-deficit era concerns the people of 
the public service and how they are managed. For government to function 
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properly, Canada needs a strong public service. If government organizations 
are to innovate and deliver high-quality services to taxpayers, the public 
service needs to attract and retain talented people. And this valuable resource 
needs to be managed properly. This takes an effective blend of tangible 
elements, like good policies and systems, and less tangible elements, like 
strong leadership. 


As I speak with senior government officials, I find much dissatisfaction with 
the current state of human resource management in the federal government. I 
have noted — as have other observers and government officials — that the 
public service is an institution under significant stress. 


Dissatisfaction with existing human resource management is also reflected in 
the interest among government officials in alternative service delivery 
mechanisms. One of the driving factors has been that present staffing, 
classification and compensation systems are too unwieldy and inflexible. The 
government needs to ensure that the rush to get “outside the system” does not 
divert attention from “fixing the system”. 


Key people management issues that need 
addressing 


e Modernized and simplified job classification 
and evaluation systems 

e Improved mobility and career opportunities 
through training, hiring and transfer processes 

e Rejuvenation and renewal of workforce to deal 
with loss of experienced staff and need for new 
skills 

e Shortages in specialized groups (computer 
specialists, engineers, statisticians, etc.) 

e More attention to the situation of employees 
who remain 


Managing the work force after downsizing is a significant 
challenge. Our audits of the government’s implementation of 
expenditure and work force reduction, reported last April, 
highlighted the need to ensure that work force capabilities 
match future demands. Early departure incentives often were 
not targeted with a clear idea of which types of employees 
would be needed in the future. The adjacent box lists a 
number of other issues that our audits identified as needing 
corrective action. 


It is troubling that these various problems still besiege us. It is 
not for lack of consensus about the need to change. It is not 
because the problems have only recently been recognized — 


the work force reductions have merely amplified issues that have existed for 


many years. 


The elimination of the pay freeze last year has removed a major irritant and 
source of malaise among public servants. A host of initiatives aimed at 
revitalizing the public service are also under way, notably La Reléve. But one 
has a sense that progress is slow, in part because there seems to be no overall 
vision of the role and makeup of the public service in a reorganized 


government sector. 


Defining that vision requires answers to some difficult questions. Should the 
public service be managed uniformly, or should different parts of the public 
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service be managed differently? What should be the respective roles of the 
central agencies and departments? How best can the principles of merit and 
political independence be protected? But difficulties notwithstanding, 
defining that vision and reforming the public service are crucial to the 
success of the new public sector model the government is trying to create. 
Unless the “people” problems are solved, the chances of successfully 
implementing other public sector reforms cannot be rated high. 


For related information, see Auditor General publications: Expenditure and Work 
Force Reductions in the Public Service (1998 Report, Chapter 1); Expenditure and 
Work Force Reductions in Selected Departments (1998 Report, Chapter 2). 


Open and Meaningful Accountability to Parliament 


One of the main responsibilities of Parliament is to hold the government 
accountable: to scrutinize the work of departments and agencies and to force 
the government to defend its policies and its handling of the public purse. To 
do this job effectively, parliamentarians need timely and relevant information 
on government objectives, expected results and actual performance. 
Information presented to Parliament also becomes 


| expect that the quality of the available to the public at large, contributing to a citizenry 


information reported will improve 


that is better informed and better able to participate in the 
government process. 


over time, as we learn by doing and 


as commitment to results and 


Significant steps forward have been taken recently in this 
area. The annual Performance Reports in the fall and the 


performance becomes more firmly spring Reports on Plans and Priorities are prominent 


established. 


examples. These are initiatives designed to improve the 
information available to Parliament and to provide greater 
opportunity for input into public policy and public management. 


It is too early to judge the success of these initiatives. I expect that the quality 
of the information reported will improve over time, as we learn by doing and 
as commitment to results and performance becomes more firmly established. 
Parliamentarians can speed the process along by making effective use of that 
information. The greater the interest they show in these new departmental 
reports, the greater will be the incentive for departments to improve their 
quality. Better information should contribute in turn to better management, 
by facilitating input from a broader cross-section of stakeholders and by 
stimulating public servants to focus on results that matter to Canadians. 


Alternative service delivery arrangements present a special 
challenge 


As the government re-evaluates its role in order to live within its means, it is 
increasingly resorting to non-departmental mechanisms to deliver services. 
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These alternative service delivery (ASD) mechanisms take many forms, from 
special operating agencies within government to partnership arrangements 
with other levels of government or non-government participants, to 
commercialization of services. While the general concept is not new, the 
government has turned increasingly to ASDs to deliver services traditionally 
provided through departments. 


ASD arrangements are generally more specialized than departments and 
enjoy more autonomy in the use of human and financial resources. 
Government control is normally exercised through more explicit framework 
agreements that specify performance targets and measures, and provide for 
less ministerial discretion. The overarching aim in all of these arrangements 
is to improve service delivery by vesting the delivery agency with greater 
flexibility and better incentives, while still maintaining adequate control over 
the use of public funds. 


Whatever the form of the agency used to deliver a 
By moving outside the traditional service, when it involves federal resources and authority 


departm ental mod el, new del ivery CA NUENS a pet to So: eer EE sTOUeR 
full and fair reporting to Parliament. By moving outside 


arrangements could dilute the traditional departmental model, new delivery 
accountability and weaken public arrangements could, if not properly set up, dilute 


service values. unless set up accountability and weaken public service values. 
Experience in other jurisdictions shows that maintaining 


properly. accountability and cohesion in these arrangements is a 
persistent challenge. 


To assist Parliament, my Office has proposed a four-point framework for 
assessing proposals for new delivery arrangements, mirroring the four key 
principles of effective governance discussed on page 11: 


e Achieving objectives. Is the new organization likely to provide better 
service? Would it meet its objectives in a cost-effective manner and 


without undue negative impacts? 


e Accountability. Does the arrangement, in its use of federal resources and 
authority, make adequate provision for accountability to the responsible 


minister and to Parliament? 


e Transparency. Does the mechanism ensure that Parliament and the 
public will receive key information? 


e Protection of the public interest. Does the arrangement adequately 
recognize and protect the essential federal public policy purpose? 
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ASDs offer the promise of more efficient and responsive delivery through 
arrangements that are more focussed, flexible and client-centred than is often 
the case in traditional departments. At the same time, they may entail 
significant changes in the roles and responsibilities of ministers and in the 
relationship of service providers with central agencies and Parliament. These 
changes, in turn, raise important questions about how to integrate ASD 
activities with the rest of the public sector, how to maintain accountability to 
Parliament and how to ensure that the public interest is being served. These 
questions need to be effectively addressed for the promise of the ASDs to be 
fully met. 


For related information, see Auditor General publications: Assessing Alternative 
Service Delivery Arrangements (Discussion Paper, June 1998); Creation of 
Canadian Food Inspection Agency (1998 Report, Chapter 12); Reporting 
Performance in the Expenditure Management System (1997 Report, Chapter 5); 
Transport Canada — The Commercialization of the Air Navigation System 

(1997 Report, Chapter 19). 


Conclusion 


Over the past few years, government spending and public sector reform have 
been driven largely by the need to reduce the huge fiscal imbalances that 
often threatened to push public finances to a crisis point. Strong incentives to 
be cost-conscious and frugal were naturally present in those circumstances. 
With the elimination of the deficit last year, these pressures have manifestly 
lessened. 


In this new environment, where paying heed to efficiency and economy may 
no longer seem as pressing, the countervailing pressures that a good system 
of public sector management and reporting can bring to bear take on new 
importance. Elements of such a system include: 


e a budget process that reminds us to take the long term into account; 


e internal governance frameworks that promote openness and good 
performance; 


e apublic service culture and incentive structures that encourage 
innovation, creativity and continued attention to the public interest; 


e accounting and information management systems that yield sound 
information for key decisions on the allocation and management of 
resources; 


e effective integration of people management with other public sector 
functions, to ensure that the work force needed to support planned 
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reforms is available and to accommodate worker concerns about those 
reforms; and 


e an open and meaningful accountability regime that gives Canadians 
assurance that their views matter and that their tax dollars are used 
wisely. 


Under the prod of tight resource constraints, much progress has been made in 
streamlining federal public administration in recent years, rendering it more 
efficient, responsive and transparent. The challenge now is to build on that 
progress, by creating new forces to sustain the momentum for improved 
performance. The challenge is difficult, but critical. Success entails more 
than giving taxpayers value for money, though it obviously does eniail that. It 
means building a better country and a healthier democratic society. 
Improving public sector performance can help restore confidence in 
government — confidence that has eroded seriously after years of 
disappointed expectations. Continued over time, it will mean increasingly 
more effective government and stronger public institutions for our children 


% 


and our children’s children. 
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Chapter Supplement: Other Recent and Ongoing Audit Issues 


Audit Assessments of Performance 


Information 


For the first time, my Office is required by legislation to 
provide an assessment of the performance information 
reported by a new service delivery agency, the Canadian 
Food Inspection Agency (CFIA). Similar legislative 
requirements are envisioned for two other proposed 
service delivery agencies: the Canadian Parks Agency 
and the Canada Customs and Revenue Agency. 


The Canadian Food Inspection Agency Act requires me 
to assess the fairness and reliability of the performance 
information included in the Agency’s annual report 
against the objectives established in its corporate 
business plan. My first assessment was included in the 
Agency’s annual report this year. The purpose of 
providing an independent assessment 1s to add 
credibility to the information reported by an 
organization about its performance. 


This first assessment provided a good learning 
experience for both the Agency and my Office. We have 
seen an increasing interest in this form of assurance in 
some provinces and other countries. As I have indicated 
before, I am willing to consider more of this type of 
work if it is found useful to Parliament and the 
government. 


I recognize that performance reporting is still 
developing in federal departments and agencies. In my 
view, more rapid progress toward better reporting is 
essential; Parliament and the public cannot be expected 
to wait long for good reporting to become the norm. 


I will continue to press for better reporting and 
accountability and to contribute, wherever possible, to 
its achievement. Parliament and ministers have an 
important role to play too: meaningful performance 
reporting in other jurisdictions has often been achieved 
only after concerted pressure from political leaders. 


For related information, see: Auditor General’s 
Assessment of Performance Information (1997-98 
Annual Report, Canadian Food Inspection Agency). 
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Ensuring Organizational Integrity in 


Revenue Canada 


Ethical conduct in the public sector is a theme of 
continuing interest for our Office. In 1995, we reported 
on work we had done on the subject of integrity in 
government. The objective was to increase the clarity of 
core principles of ethical decision-making. The study 
proposed a framework for ethics in government based 
on the premise that “public service is a public trust.” 


In September of this year, we reported on how Revenue 
Canada promotes integrity among its employees. Our 
audit found that the Department recognizes the 
importance of a strong ethical culture to its success, and 
has adopted a number of measures to nurture a working 
environment supportive of an ethical culture. In 
addition to common procedures for investigating 
misconduct, disciplining offenders and correcting 
identified deficiencies, these measures include: 


e incorporation of integrity and fairness among the 
principles in the Department’s vision statement; 

e dissemination of written codes of conduct to all 
staff; and 

e training in the behaviour expected of employees. 


These measures are consistent with the framework 
proposed in our 1995 study, and form a strong 
foundation for the promotion of integrity in Revenue 
Canada. 


The audit also identified areas for improvement to 
reinforce that foundation. In particular, the codes of 
expected conduct could be made clearer and their 
rationale explained. As well, Revenue Canada could 
provide an ethics advisor to be available for private 
consultation and to act as a champion for measures 
aimed at promoting integrity in the Department. 


Having ethical standards in place is not sufficient to 
ensure ethical performance, but it is a necessary step. 
Revenue Canada’s initiatives in this area provide 
examples that other departments should consider 
emulating. 


For related information, see Auditor General 
publications: Promoting Integrity in Revenue Canada 
(1998 Report, Chapter 15). 
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Examination of the Requirement to Report 


Specified Foreign Property 


On 19 December 1997, the Governor General in 
Council, on the recommendation of the Minister of 
Finance and the Minister of National Revenue, asked 
me to examine the requirement to report specified 
foreign property under section 233.3 of the Income Tax 
Act. My report on the examination was tabled in the 
House of Commons on 5 June 1998. 


We concluded that the requirement to report specified 
foreign property is an appropriate mechanism, within an 
overall strategy to enhance compliance with the laws 
that require reporting of foreign-source income and also 
to provide Revenue Canada with information to verify 
taxpayers’ self-assessments. We also examined 
alternative mechanisms and concluded that none would 
be as effective as the existing requirement. It is with 
some disappointment, therefore, that I note the 
government has decided to implement a “check the 
box” alternative (which simply requires taxpayers to 
answer yes or no to a series of questions, without 
providing details about their assets). However, the 
government did accept most of our other 
recommendations. 


This examination also provided us with a unique 
opportunity to seek the views of Canadians and tax 
experts abroad on the issue. Many who sent us 
submissions expressed serious and deep-felt concerns 
about privacy, economic and investment issues. We 
studied these concerns carefully and found that many 
were related most likely to the rates of taxation, the 
requirement to pay tax on worldwide income and a poor 
understanding of Canada’s tax system, rather than to the 
reporting requirement we were asked to examine. 


For related information, see Auditor General 
publication: Examination of the Requirement to Report 
Specified Foreign Property Under Section 233.3 of the 
Income Tax Act (1998 Report to the Ministers of 
Finance and National Revenue). 
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Progress on the Federal Science and 
Technology Strategy 


In a series of reports for the better part of this decade, 
my Office has been encouraging the government to 
improve the management of its science and technology 
activities. The federal government unveiled its new 
science and technology strategy in March 1996, aimed 
at promoting scientific inquiry and applying technology 
in the best interests of Canadians. Our concern at this 
early stage is that the Strategy may lose its initial 
momentum, as happened with similar efforts in the past. 


For this reason, we reviewed the government’s progress 
to date in implementing the Strategy. In doing so, we 
kept in mind the messages of our previous work, 
namely, the need for mission-driven and results-based 
research, clear accountability and a focus on 
performance. 


Our audit, presented in Chapter 22 of this Report, found 
that the government has been slow in establishing a new 
governance system for its science and technology 
activities and that it has not yet acted fully on its 
commitments under the Strategy. As a result, the 
management regime set up to oversee federal science 
and technology activities is not yet working as intended 
— that is, ensuring clear priorities, co-ordinated efforts 
and full reporting on performance. 


To ensure that the Strategy stays on track, the 
government should set out clearly what still needs to be 
done and provide the necessary leadership to have it 
done. In an information-based world, getting our 
science and technology programs right is of paramount 
importance. 


For related information, see Auditor General 
publications: The Federal Science and Technology 
Strategy — A Review of Progress (1998 Report, Chapter 
22); Federal Science and Technology Activities — 
Follow-up (1996 Report, Chapter 15); Overall 
Management of Federal Science and Technology 
Activities (1994 Report, Chapter 9); Management of 
Departmental Science and Technology Activities 
(1994 Report, Chapter 10). Other publication: Science 
and Technology for the New Century: A Federal 
Strategy (March 1996). 
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Millennium Bug Still at Large 


The Year 2000 problem, also known as Y2K and the 
millennium bug, refers to the potential for computer 
system failures as the year 2000 approaches. The 
problem stems from the common practice among 
computer programmers in the past (adopted for 
cost-saving reasons) to represent a year by two digits 
only. Computers so encoded will be unable to 
distinguish the year 2000 from the year 1900. That 
could cause them to shut down or otherwise 
malfunction. The potential for widespread disruption 
from such malfunctions is significant. 


In 1997, I expressed concern, based on the results of an 
audit we had just completed, that government progress 
in preparing its systems for the Year 2000 had been 
slow. In a follow-up audit this year, we found that the 
pace of work on Year 2000 projects has accelerated and 
much progress has been made. But much remains to be 
done. 


The Treasury Board Secretariat has been conducting 
surveys of systems in departments and agencies that 
support many key, mission-critical functions across 
government. Its most recent survey, completed in June 
1998, showed that many systems were falling behind an 
already tight schedule. Our audit this year found that 
three of six mission-critical functions we examined 
remained at risk. | am very concerned that systems 
supporting many government-wide critical functions are 
at risk of not being Year 2000 compliant on time. 


The Year 2000 problem is one to which solutions cannot 
be postponed. The government should continue to rank 
it as a top priority, intensify efforts to update systems to 
make them Year 2000 compliant, and develop 
contingency plans to ensure that essential government 
services are maintained as the Year 2000 rolls in. 


For related information, see Auditor General 
publications: Preparedness for Year 2000: 
Government-Wide Mission-Critical Systems (1998 
Report, Chapter 20); Information Technology: 
Preparedness for Year 2000 (1997 Report, Chapter 12). 
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A Strategic Approach to Sustainable 


Development 


Organizations in both the private and public sectors are 
under increased pressure to improve their environmental 
performance and to contribute to sustainable 
development. The federal government has at times 
driven action in the private sector through its regulatory 
role; at other times it has played catch-up with practices 
that have taken root elsewhere. 


As the largest single enterprise in Canada, the federal 
government is in a unique position to lead by example, 
by improving its environmental performance in areas 
like procurement, waste management, and the use of 
water, energy and land. 


The government’s impact on the environment, however, 
extends beyond its role as employer and consumer. Its 
major impact flows from the large number of policy and 
program decisions that influence the way individual 
producers and consumers affect the environment. The 
high clean-up costs and environmental damage at the 
Sydney Tar Ponds in Cape Breton illustrate some of the 
consequences of a lack of environmental foresight. 


Twenty-eight federal organizations have now prepared 
sustainable development strategies and tabled them in 
the House of Commons. Through those strategies, 
departments are being challenged to take 
environmental, economic and social considerations into 
account more systematically across the board — in 
their policies, their programs and their day-to-day 
operations. 


A year ago I said that a key challenge facing the federal 
government was to go beyond the rhetoric of 
sustainable development, and to focus on its practical 
implementation. The Commissioner of the Environment 
and Sustainable Development will be reporting on 
whether departments did what they said they would do. 


For related information, see Auditor General 
publication: 1998 Report of the Commissioner of the 
Environment and Sustainable Development. 
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Opportunities for Cost Savings 


Improving the efficiency and effectiveness of 
government activities and cultivating a healthy regard 
for economy are objectives central to our mandate and 
to the work this Office does. One way of promoting 
these objectives is by highlighting specific opportunities 
for cost savings. The following are examples taken from 
this year’s Report: 


e National Defence — Equipping and Modernizing 
the Canadian Forces — Chapter 3. As part of its 
renewal efforts, National Defence is seeking ways to 
improve productivity and lower costs of operations. 
One option being considered is to reduce the 
frequency of personnel transfers. According to 
departmental estimates, this could save about 
$100 million per year. 


e Federal Laboratories for Human and Animal Health 
Building Project — Chapter 7. This Winnipeg-based 
project could have been delivered for at least 
$5 million to $10 million less than the estimated 
$176 million it cost to complete. 


e 1998 Report of the Commissioner of the 
Environment and Sustainable Development. 
Common indicators for reporting environmental 
performance could facilitate comparisons among 
departments and provide useful benchmarks for 
bringing about cost reductions. According to one 
recent study, implementing energy conservation 
measures government-wide could yield net savings 
of $29 million per year by 2005. 


Often our audits also find opportunities for cost savings 
that, while significant, are not easily quantifiable. For 
example, in Chapter 16 Management of the Social 
Insurance Number, we report that there are nearly four 
million more active Social Insurance Numbers than 
there are Canadians aged 20 or older. The Social 
Insurance Number is used as an identifier to verify 
entitlements to a wide array of social benefits and to 
assess taxes owed. The amounts involved are so huge 
(approximately $100 billion in annual payments for all 
levels of government) that even a relatively low 
incidence of fraud and abuse could represent significant 
costs to governments. 
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Expenditure and Work Force 
Reductions in the Public Service 


Chapter 1 — Main Points 


1 Expenditure and work force reductions have affected the public service and public servants for more than 
a decade. In 1995, the government launched Program Review, an initiative aimed at rethinking and realigning 
government programs in light of fiscal restraints. 


1.2 To better manage expenditure and work force reductions, central agencies moved from imposing, 
directing and prescribing to providing leadership, guidance and support to departments. For example, instead of 
adopting the “one size fits all” approach, Treasury Board Secretariat proposed a management and accountability 
framework to departments and allowed incentive programs to take into account the differences in their context and 
operational environment. 


1.3 Program Review was a success from many perspectives. The deficit has been reduced and personnel costs 
have decreased for the first time in more than a decade. In spite of a difficult climate, reductions were 
implemented without major work disruptions; and financial departure incentives have been effective in 
minimizing layoffs. 


1.4 The cost of achieving work force reductions was high, however, not only in dollar terms but in human 
terms as well. For example, between 1992 and 1997 the government paid incentives to some 46,000 public 
servants, military and RCMP members to leave or retire. Furthermore, work force reductions have amplified a 
number of human resource issues, such as the need for rejuvenation and renewal and the need to modernize and 
simplify human resource systems. Many studies and our own audit work indicate that the public service is an 
institution under significant stress and that strong actions are now required. 


1.5 Significant improvements were made in the management framework put in place by central agencies. 
Nonetheless, more efforts are needed to improve the management of work force reductions — for example, in 
critical occupational groups where there might be surpluses in some departments but shortages in others. 


1.6 There is a need to improve accountability to Parliament for expenditure and work force reductions, and 
for the related costs. Roles, responsibilities and accountabilities of central agencies and departments in the 
management of work force reductions also need to be clarified. 
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Expenditure and Work Force 
Reductions in Selected 
Departments 


Chapter 2 — Main Points 


2.1 As a result of a lengthy period of fiscal restraint and recent expenditure reduction programs, some federal 
government agencies have been abolished, some departments merged and others reduced in size. It is estimated 
that the public sector work force will have been reduced by approximately 45,000 employees between April 1995 
and March 1998. ; 


22 We selected seven departments to examine how and the extent to which expenditure reductions have been 
implemented. These organizations, representing over 170,000 employees, differed in size, structure and reduction 
targets. 


23 Generally, departments we reviewed met their 1995—96 and 1996-97 expenditure and work force 
reduction requirements. Given progress to date, we expect that most of them will meet their reduction targets for 
the remaining two years of Program Review. 


2.4 We found that ministerial commitment and departmental leadership were evident in setting direction and 
attaining momentum. Factors such as the size of the reduction, readiness for change and the time available to plan 
had an effect on departments’ strategic planning for reduction. Departmental implementation of reduction was 
generally rigorous and employees were well informed both prior to and during the reduction process. 


2.5 Although most reduction targets were met, and overall compliance with work force reduction policies had 
improved since our 1992 Report chapter, Payments to Employees under the Work Force Adjustment Policy, we 
found questionable actions in some departments. The need for incentive packages was not always well researched 
and there was a tendency to call for volunteers prior to identifying positions surplus to requirement. Best practices 
suggest that an organization should conduct a thorough cost/benefit analysis of alternatives; target incentives to 
areas where they are needed most; and identify the extent to which there is a need to call for volunteers. 


2.6 Departments focussed primarily on meeting reduction targets in a timely manner. Less emphasis was 
placed on cost management. Such factors as the initial lack of necessary financial and human resource 
information, departments not having to fund entirely the departure incentives they provided, and an unclear 
Treasury Board Secretariat accountability structure for departmental work force reduction costs contributed to a 
general lack of cost consciousness in departments. 


2.7 Throughout work force reduction, departments treated departing employees in a humane and sensitive 
manner. Departments paid less attention generally to those who remained and to their concerns regarding the loss 
of experienced and qualified colleagues, the level of workload per employee and future departmental direction. 
Departments have now entered a period of transition that will require not only adjusting organizational structures 


and operations but also paying particular attention to ensuring that work force capability meets future operational 
demands. 
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National Defence 


Equipping and Modernizing the 
Canadian Forces 


Chapter 3 - Main Points 


Sal A modern, multi-purpose, combat-capable force able to “fight alongside the best, against the best” is 
necessary to fulfil Canada’s defence policy commitments, although the policy does not require the Canadian 
Forces to possess every component of military capability. 


3.2 According to Department of National Defence officials, Canada’s White Paper commitments are defined 
by the defence capabilities of the Canadian Forces. We found many equipment deficiencies that limit capabilities. 


33 If the status quo persists, the Department’s available capital funding may not be sufficient to equip and 
modernize the force that National Defence is currently planning. Officials told us that they expect to be able to 
increase the capital portion of the budget. Nevertheless, they anticipate that hard choices may have to be made. 


3.4 Personnel, operations and maintenance costs are rising, which 1s further reducing the portion of the 
budget available for capital equipment. 


a We found that the Department does not have an adequate policy framework to direct the almost 
$1.4 billion it spends each year on equipment modernization. It does not yet have in place fully developed 
operational scenarios to guide planners, or performance information that identifies gaps. 


3.6 Some other countries — notably the United States and New Zealand — have taken the lead in the way 
their government budgeting and management systems actively manage defence policy and resources. 
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National Defence 


Buying Major Capital Equipment 


Chapter 4 - Main Points 


4.1 National Defence plans to spend almost $6.5 billion over the next five years to purchase equipment that 
Canada’s armed forces need to carry out their assigned tasks. As Chapter 3 indicates, the Department’s available 
capital funding may not be sufficient to equip and modernize the force that is currently planned. 


4.2 This chapter reports our findings on the management of six major capital projects with a total value of 
$3.3 billion. 


4.3 All six projects we examined are likely to meet their contract cost and performance objectives. For 
example, the Eryx anti-tank missile system and the Coyote reconnaissance vehicle appear to perform especially 
well and have attracted the interest of foreign armed services. Nevertheless, we are concerned about several of the 
management practices we observed. 


4.4 We found that defence policy allows wide latitude in the level of equipment capability required. We 
noted in the projects we examined that affordability constraints resulted in only low-end capability being 
purchased, limitations in the number purchased, or both: 


e the Maritime Coastal Defence Vessel is acquiring two sets of mechanical minesweeping gear for 12 
ships effective only against some types of mines, and has not acquired some of the equipment 
necessary to patrol effectively in darkness and poor visibility; 


e the Leopard C1 tank is being fitted with only an improved sight, although the army had considered a 
complete upgrade, including the gun and armour, to be the “minimum viable” option; 


e the Griffon helicopter cannot meet the army’s original lift and communications requirements; 
e fewer Eryx missiles and Coyote vehicles have been purchased than were originally intended. 


4.5 We are concerned at the extent to which the Department relies on professional judgment in making 
complex purchase decisions. Management did not conduct adequate analyses to justify its spending decisions for 
most of the projects we examined. Tactical studies often did not reflect the way officials said they actually 
planned to employ equipment, were done too late to influence decisions, produced results that contradicted the 


purchase decision, were undertaken by contractors who had an interest in the Department’s decision, or were not 
done at all. 


4.6 In three cases, the Department considered only a single option. In other cases, the options analyses were 
inadequate. 
4.7 Only one project of the six fully met our expectations for a rigorous risk management process. No project 


we examined has suffered major problems so far because of this, but we identified several instances where better 
risk management might have improved project delivery. 
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Chapter 4 - Main Points (cont’d) 


4.8 Although test and evaluation processes were satisfactory in most cases, for some of the projects that 
involved commercially based designs, the equipment was not tested under actual operating conditions until after it 
had been introduced into service. Problems have since been discovered that have affected the operational 
capability of some of the equipment. The Department is currently working to resolve these problems. 


4.9 The Department began a major management renewal program in 1994 that included the capital 
acquisition process. Because it considered other areas to be more important, it deferred work on capital equipment 
acquisition and it does not yet have an implementation plan. The Department has begun work on the plan, 
including three pilot projects. Because equipment acquisition is a government-wide process, the Department is 
working with Public Works and Government Services Canada, Industry Canada, and the private sector to improve 
government practices. We believe the departments concerned must develop a plan with definite dates and 
milestones to complete their reform process. 
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Revenue Canada, 
Department of Finance and 
Department of Justice 


Interdepartmental Administration of 
the Income Tax System 


Chapter 5 — Main Points 


5.1 Three departments play a key role in administering the income tax system. The Department of Finance is 
responsible for the formulation of tax policy and the introduction of new tax legislation, Revenue Canada oversees 
the administration of tax laws and the Department of Justice provides legal advisory services and litigation 
services to both Finance and Revenue Canada. 


5.2 Efficient and effective relationships among Revenue Canada, the Department of Justice and the 
Department of Finance are essential for the smooth functioning of the income tax system. This audit was 
undertaken because our past work had raised some concerns about the management of risk to the tax base. Our 
current examination found that the departments have taken many important measures to improve their 
relationships. We have been told that the proposed change of Revenue Canada to a new Canada Customs and 
Revenue Agency will not affect their relationships. 


5.3 An important aspect of managing the risk to the tax base is the identification and correction of legislative 
deficiencies. We found that Revenue Canada and the Department of Justice regularly bring deficiencies to the 
attention of the Department of Finance. However, the correction of legislative deficiencies is left to the discretion 
of the Department of Finance. 


5.4 As all three departments are key players in managing the risks arising from legislative deficiencies, it is 
important that they all be involved in deciding which deficiency should be given priority. 


SYS) The management of tax litigation risks has been improved. However, Revenue Canada and the 


Department of Justice need to strengthen the partnership arrangement for planning litigation needs and managing 
the risks. 


5.6 Settlements of income tax disputes resolved through consent judgments have been made on a mix of law 
and facts. Revenue Canada should provide transparency in the settlement agreements it undertakes. As an 
example, United States income tax law provides public access to settlement agreements, with privacy protection 
of the taxpayer’s identity. 
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Population Aging and 
Information for Parliament: 
Understanding the Choices 


Chapter 6 - Main Points 


6.1 Canada today is sitting on a very favourable demographic structure, with a historically low ratio of youth 
and elderly to the working-age population. In the coming few decades, this situation will be radically transformed. 
By the second decade of the next century, when the leading edge of the baby boom generation reaches normal 
retirement age, the growth in Canada’s elderly population will accelerate while that of the working-age population 
will slow to a crawl. 


6.2 This demographic shift has the potential to affect government finances in a significant way. An aging 
population puts pressure on government spending through higher pension payments and increased demands for 
health care services. Also, unless there are significant changes in the patterns of work, this demographic change 
will reduce labour force growth dramatically, which in turn would slow down economic growth and, with that, the 
growth in government revenues. This combination could put enormous pressures on our public finances when the 
full impact of this demographic change is felt by the second decade of the next century, particularly if Canada’s 
debt burden and tax levels remain high. 


6.3 The government is well aware of these pressures; during consultations on reforming specific programs 
like the Canada Pension Plan, it provided detailed projections of the effects of demographic forces on CPP 
finances. This longer-term information not only helped inform public debate, but also motivated acceptance of the 
need for change. To a lesser extent, as government began discussing reforms to Old Age Security and the 
Guaranteed Income Supplement (the Seniors Benefit), it again provided longer-term projections of the financial 
impacts. 


6.4 Yet what it has done to highlight the impact of demographics on the financial health of individual 
programs, it has failed to do for the financial health of the government as a whole. Consequently, parliamentarians 
are left to make annual financial decisions, many of which have lasting consequences, without a macro 
perspective — a summing up of the financial impacts that demographics can potentially have on the government’s 
long-term financial health. 


6.5 The importance of appropriate information to help parliamentarians understand the government’s 
financial condition is clearly a responsibility and a theme on which we have reported over the last six years. This 
chapter, illustrating the feasibility of preparing such longer-term information and the significance of that 
information, is a continuum of the “Information for Parliament” chapters in our 1993 and 1995 Reports. The 
particular contribution of this chapter is its focus on the role of demographics in assessing the government's 
financial condition. 
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Federal Laboratories for 
Human and Animal Health 
Building Project 


Chapter 7 - Main Points 


re The new Federal Laboratories for Human and Animal Health in Winnipeg contain some of the most 
advanced diagnostic, research and training facilities of their kind in the world. The facility is the first in the world 
to combine laboratories concerned with both human and animal diseases, and the first in Canada capable of 
handling the most dangerous viruses known. The facility is to be used by Health Canada and the Canadian Food 
Inspection Agency. 


7.2 International experts involved in reviewing the design and construction of the facility have noted that it is 
extremely secure with regard to containment of biohazardous materials. 


es Despite a lengthy history of project proposals by Agriculture and Agri-Food Canada and Health Canada 
to construct new laboratory facilities, the Winnipeg project was undertaken without adequate analyses of existing 
laboratory capacity. The Winnipeg facility currently has space that exceeds its planned program needs and the 
amount approved by Treasury Board, and that has added to a national inventory of surplus laboratory capacity. 


7.4 Health Canada and the Canadian Food Inspection Agency have informed us that they recognize the 
importance of ensuring that the facility is fully used, including the need to explore potential cost-recovery 
opportunities. To date, however, a comprehensive business plan and strategy for achieving these aims has not been 
developed. 


fi) In our opinion, the project is approximately 12 months behind schedule and could have been delivered 
for at least $5 million to $10 million less. In addition, the approved project budget of $141.8 million does not 
accurately reflect the project’s total cost to the taxpayer, currently estimated at $176 million. The proposed project 
budget did not identify all project costs, as required by Treasury Board policy. 


7.6 We identified several areas where Public Works and Government Services Canada can improve its 
practices in selecting and managing consultants on future complex projects like this one. We also noted that 


shared authority and accountability require careful management to mitigate risks of project delays and added 
costs. 


Teil As we have noted in reviewing other government projects, the current “build [up] to budget” approach, 
the lack of systemic incentives for seeking cost savings within approved budgets, and the absence of a 


departmental culture that supports such aims do not encourage the parties to identify and realize potential cost 
savings. 


7.8 The new facility is a potential source of pride for all Canadians as it contributes to the global effort to 
identify and combat human and animal infectious diseases. Due to the nature of its programs, the Winnipeg 
laboratories will be expensive to operate and maintain. The challenge for Health Canada and the Canadian Food 
Inspection Agency will be to ensure that the full potential of this new facility is achieved. 
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Department of Finance 


Effectiveness Measurement and 
Reporting 


Chapter 8 — Main Points 


8.1 In addition to functions that include preparing the Budget and providing economic policy advice to the 
government, the Department of Finance has responsibilities related to a number of policies and programs. The 
Department’s 199798 planned expenditures of $66 billion account for 43 percent of planned federal expenditures. 
The Department is also responsible for federal tax policy, including preferential tax provisions that are intended to 
lead to the achievement of a variety of government policy objectives. 


8.2 The significance of the Department’s policy and program responsibilities underlines the importance of its 
obligation to make known to Parliament and to the public, in a clear and meaningful manner, the nature and extent 
of those responsibilities, the objectives involved and the results achieved relative to those objectives. 


8.3 Improvement is required in providing Parliament with clear and consistent statements of objectives and 
of the Department’s policy and program-related responsibilities. For many policies and programs, the Department 
shares responsibilities with other departments and agencies. We found a general lack of clarity in documents 
directed to Parliament about the other departments and agencies involved and the manner in which responsibilities 
are shared. 


8.4 We found gaps in reporting to Parliament on results achieved relative to objectives. Our findings in this 
audit are consistent with the findings of a number of audits of individual policies and programs undertaken by this 
Office over the past six years. The picture that emerges is one of known shortcomings, and insufficient attention to 
measuring effectiveness and reporting results to Parliament and the public. 


8.5 Legislative provisions governing a number of the Department’s policies and programs have been the 
subject of review by Parliament to consider whether the provisions should be changed to improve performance 
and to enable policies and programs to be adapted to new circumstances. Such legislative reviews are valuable in 
their own right; they are not, however, a substitute for systematic measurement and reporting of effectiveness 
information by the Department. We believe legislative reviews would be enhanced by Parliament having more 
regular and timely access to information on the results that policies and programs have achieved. 


8.6 Planning for effectiveness measurement and reporting is not sufficiently linked with the Department’s 
corporate planning process. The Department’s current approach results in an effectiveness measurement function 
that is partial in coverage and not clearly integrated with the overall corporate management structure. In view of 
the limited resources allocated specifically to the function, we are concerned about the Department’s capacity to 
meet its obligations to account to Parliament for results achieved. 
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Reporting Government Financial 
Results: The Importance of 
Complying with Objective 
Accounting Standards 


Chapter 9 - Main Points 


9.1 The 1980s and early nineties have seen major advances in financial reporting by sovereign governments, 
with Canada among the leaders. However, over the past two years the Government of Canada has begun departing 
from both objective accounting standards and its own stated accounting policies for certain transactions. 
Moreover, the 24 February 1998 Budget indicates that a similar departure will occur in 1998. 


9.2 In 1996, transitional assistance for harmonizing PST with GST was included in the deficit even though 
provinces had not yet agreed to the harmonization by the end of the year. In 1997, funding of the Canada 
Foundation for Innovation was included in the deficit even though the Foundation did not exist by the end of the 
year. In 1998, the 24 February 1998 Budget indicates that funding of the Canada Millennium Scholarship 
Foundation will be included in financial results whether or not the Foundation exists by the end of the year. 


9.3 As auditors of the government’s financial statements, we are deeply concerned over this recent disregard 
by the government for objective accounting standards in reporting its financial results to Parliament and to 
Canadians. This is not merely a technical disagreement between an accountant and an auditor. Objective 
accounting standards are accepted and identifiable standards that are established by an arm’s-length 
standard-setting body — the Canadian Institute of Chartered Accountants (CICA). Business firms cannot depart 
from objective accounting standards established by the CICA, to hide losses or to hide profits. Parliamentarians 
should expect no less from the government. 


9.4 Meeting the significant challenges of the next decade will require, among other things, credible annual 
reports by the government on its overall financial health. In our view, departures from CICA standards and the 
government’s own accounting policies must cease if credibility is to be restored to the reports that 
parliamentarians and others need. 


oS Accordingly, we have recommended that the Government of Canada ensure that its accounting policies 
and practices conform with recommendations issued by the CICA’s Public Sector Accounting and Auditing Board. 
{mplementation of this recommendation will require continued diligence by the Public Accounts Committee of the 
House of Commons in reviewing the government’s annual financial statements. 


9.6 Although this chapter calls Parliament’s attention to inappropriate accounting, there are related concerns 
that we have and will continue to report separately. They deal with accountability and parliamentary oversight for 
entities such as the Innovation Foundation and the Millennium Scholarship Foundation, and with whether or not 
such entities are, in substance, operating at arm’s length from the government. 
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Canadian Human Rights Commission 


Human Rights Tribunal Panel 


Chapter 10 - Main Points 


10.1 In 1977 Parliament established the Canadian Human Rights Commission and the Human Rights Tribunal 
Panel to resolve human rights complaints quickly, impartially and expertly. The Commission investigates 
complaints and, if it decides that an inquiry is warranted, forwards the complaint to the Tribunal for adjudication. 
This model was chosen as the alternative to the formal processes of the courts. However, the approach that has 
evolved is cumbersome, time-consuming and expensive. 


10.2 Between 1988 and 1997, the Commission received about 7,450 signed complaints, excluding pay equity 
complaints. It made final decisions on about 6,550 of the 7,450 signed complaints. The Commission dismissed or 
did not proceed with about two thirds of these complaints. About six percent of complaints were forwarded to the 
Tribunal for inquiry. 


10.3 Although major efforts have been made to improve the handling of complaints, it still takes a long time 
for a complaint to be reviewed — an average of about two years for the Commission and about a year for the 
Tribunal. In 1997 almost one half of the Commission’s 900 open cases were still being investigated one year after 
the signing of the complaint. 


10.4 The Commission tries to settle complaints through conciliation and the Tribunal also tries to mediate 
complaints after the Commission forwards them for inquiry. Because the Commission has its own settlement 
objectives, it is not a neutral conciliator. Since 1996 about 18 percent of complaints have been settled through 
early resolution or conciliation. The Commission took an average of about 45 months to reach a final decision 
when complaints were investigated and then sent to conciliation. 


10.5 The Commission and the Tribunal are operating in a complex environment that poses major risks. The 
grounds for making a complaint are increasing and concepts of discrimination are becoming more complex. The 
environment also is more litigious and contentious. The Commission’s total funding has been reduced from a high 
of $16.15 million in 1992-93 to $14.8 million in 1998-99. The number of investigators also has been reduced 
from a high of 37 to 25. Further, the Commission is experiencing a high turnover of investigative staff, 
necessitating the reassignment of cases to other investigators. We also are concerned that the Commission is not 
consistently adhering to some of its key investigative standards. 


10.6 We note that the Minister of Justice has stated that a broad review of the Canadian Human Rights Act 
will be undertaken. The Commission has welcomed this proposal. The December 1997 Third Report of the Senate 
Standing Committee on Legal and Constitutional Affairs also called for a comprehensive assessment. 


10.7 We agree that a fundamental review by Parliament is needed and recommend that the government 
identify and present to Parliament an integrated set of specific measures to increase the effectiveness of addressing 
human rights complaints. Possible measures are summarized in the concluding section of this chapter. 
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Agriculture and Agri-Food Canada 


Cash Advance Program 


Chapter 11 — Main Points 


11.1 The Department’s cash advance program is one vehicle the government makes available to producers of 
storable crops to provide additional flexibility in making marketing decisions. It consists of a short-term loan 
made available at harvest, a time when financial obligations of producers are often at their peak. The financing 
provides a bridge for producers until they market their crop and begin to earn revenue. In recent years, the annual 
amount of these loans has ranged from $730 million to over $1.2 billion. 


11.2 The program has been very well received. Producers we contacted liked the availability and predictability 
of cash flow at harvest time, and the interest-free feature of the program. Many of the large agricultural assistance 
programs of the past have ended, and producers see this program as an important sign of continuing government 
support for the sector. 


11.3 In 1993, the Department’s Review Branch completed an evaluation that reached some positive 
conclusions about the program, but also raised some significant concerns, particularly with the program’s 
interest-free nature. It also concluded that there was no firm evidence that the program contributes to orderly 
marketing. 


11.4 In 1997 new legislation, the Agricultural Marketing Programs Act (AMPA), was passed that updated and 
consolidated the two previous cash advance programs into one, and made permanent in legislation the interest-free 
provision. The rationale for providing advances interest-free was not made clear by the government, despite the 
evaluation’s finding that a clear objective for this component was needed. 


11.5 Under the new legislation, a review of the provisions and operations of the Act must be completed in the 
fifth year of operation, which begins in 2001. The 1993 evaluation furnished important information about the 
program’s effectiveness; however, as in any study, some questions remain unanswered that we believe should be 
addressed in the coming review. Among others, these include whether producers indeed face a significant problem 
of access to credit at harvest time and whether losses arising from the government’s guarantee on these loans are 
being well managed. 


11.6 We identified limitations in how the cash advance evaluation findings were publicly reported. Similar 
limitations were noted in the dissemination of other studies produced by departmental branches on the impacts 
and effectiveness of programs. In our view, a number of characteristics of the Department, its legislation and 
stakeholder community create a strong case for more actively alerting outside parties to the existence and findings 
of effectiveness studies. Some areas of the Department are indeed taking steps in this direction. 
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Creation of the Canadian Food 
Inspection Agency 


Chapter 12 - Main Points 


12.1 The federal government is considering the use of alternative service delivery (ASD) arrangements in 
order to improve service delivery. The Canadian Food Inspection Agency (CFIA) was the government’s first 
legislated service agency; it was created as a departmental corporation and became operational 1 April 1997. Its 
primary responsibility is to contribute to food safety, plant and animal health as well as to facilitate trade in all 
areas by providing inspection and related services. 


12.2 There was a clear rationale for creating the Agency, including a 30-year history of studies related to 
reorganizing the federal food inspection system and the government’s commitment to a new and better way of 
delivering services. Senior officials of the three federal departments then involved in food safety inspection 
supported the initiative to create a single food inspection service. This initiative illustrated that change can be best 
introduced and sustained when there is a clear rationale that has the support of senior officials. 


12.3 The designation of a structured and independent team was crucial to managing the move to agency status. 
The team’s work was separate from the ongoing business of inspections and quarantine activities, so those services 
could continue without interruption throughout the creation of the CFIA. 


12.4 One of the driving forces behind the move to ASD arrangements is their greater flexibility in human 
resource management. With the move outside the Public Service Employment Act and the change to separate 
employer status, the shift to a new human resources regime was one of the Agency’s greatest challenges. The 
complexities entailed in creating a new human resource framework as a separate employer need to be recognized 
and planned for early in the change process. 


12.5 In an area of shared responsibility such as food safety, roles need to be carefully defined and mechanisms 
established to resolve unanticipated problems. The establishment of an accountability framework was a key 
concern during the creation of the CFIA. Specific responsibilities for food safety were assigned to the Minister of 
Agriculture and the Minister of Health. Some operational details were still being worked out as new issues arose 
during the first year of operations. 
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National Energy Board 


Chapter 13 - Main Points 


13.1 The National Energy Board Act has not been amended significantly since it established the National 
Energy Board (NEB) in 1959. While the NEB has been given some new responsibilities in areas such as frontier 
activities, its role has not changed much since its creation. However, its modus operandi has changed significantly. 
Through regulatory changes, the Board has moved from an interventionist approach to reliance on market-based 
forces, industry self-regulation and negotiated toll structures. During this period of evolution, the NEB has moved 
its operations from Ottawa to Calgary, downsized its staff complement by 35 percent and reorganized itself 
internally for greater efficiency. However, it has not objectively measured the extent to which it is being effective 
and relevant. 


13.2 The Board has recognized that the aging of the Canadian pipeline system poses potential threats to public 
safety and the environment. The number of reported pipeline incidents has increased in recent years. A risk 
assessment methodology is being developed that should improve the targeting of inspections and safety audits. 
However, deficiencies in information management will need to be addressed. 


13.3 The Board’s ability to fully meet its environmental regulatory responsibilities is at risk due to shortfalls in 
its environmental inspection program — namely, in setting priorities and scheduling inspections and in inspection 
practices, which are too informal and unstructured. 


13.4 Since 1991, the NEB has recovered most of its costs from industry. No change has yet been made to its 
cost allocation method, despite some concerns expressed by stakeholders about its fairness. The NEB has settled 
for simplicity at the expense of cost causality. The Board does not have a cost accounting system; it needs to 
recover costs in a way that is more equitable and acceptable to stakeholders. 


13.5 The NEB is moving toward management for results, by adopting a new accountability framework and 
changing its style of decision making. While it is too early to evaluate the results of this initiative, we noted that 
significant opportunities exist to better link its human resource management activities to its vision of the future. In 
addition, the NEB will need to state and report clearly what it wants to achieve, monitor its performance on a 
regular and systematic basis and better link the results achieved to the costs. 
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Indian and Northern Affairs Canada 


Comprehensive Land Claims 


Chapter 14 - Main Points 


14.1 Comprehensive land claim settlements are modern treaties that are significant not only to Aboriginal 
communities but also to all Canadians. Settlements involve the payment by the federal government to Aboriginal 
groups of almost two billion dollars and the clarification of various rights, including full ownership of over a 
half-million square kilometres of land. The first modern negotiated settlement of a comprehensive land claim was 
signed in 1975, and the most recent settlement was reached in July 1997. As of July 1997, 12 claims had been 
settled. Actual and potential claims involving over 200 First Nations are still to be settled. 


14.2 The Government of Canada has chosen to settle these claims by negotiation rather than litigation or other 
means. Indian and Northern Affairs Canada is the lead federal representative in negotiations, with participation of 
other federal departments as appropriate. Other parties include the territorial governments, provincial 
governments, and the claimants. Although the Department bears significant accountability for results, it is not the 
only party that should be accountable; the good will and political resolve of all participants are needed in order to 
achieve appropriate settlements. 


14.3 The government has reported that negotiated settlements establish certainty to land title and access to 
land and resources. However, removing uncertainty involves much more than signing an agreement. Among other 
things, successful implementation of settlements is critical to achieving the intent of these treaties. 
Implementation involves hundreds of projects with joint or separate participation of the parties. We found 
deficiencies in implementation, including inadequate or non-existent implementation plans, and the need to 
improve monitoring, reporting and evaluating. 


14.4 Indian and Northern Affairs Canada has not demonstrated that it has always exercised adequate rigour in 
determining the nature and amount of assets that may ultimately be included in final settlements, including such 
considerations as the amounts of cash, land and other resources. 


14.5 Settlements can take over 20 years to reach. Delays have had many causes. In our view, these long time 
frames do not contribute to cost effectiveness, and may result in less desirable outcomes for all concerned. 
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Promoting Integrity in 
Revenue Canada 


Chapter 15 — Main Points 


15.1 The success of Revenue Canada’s programs depends largely upon voluntary compliance with tax 
requirements by individuals and businesses. We believe that one of the many factors affecting the likelihood of 
their voluntary compliance is their perception of the conduct of Revenue Canada’s staff. An organization like 
Revenue Canada, with its decentralized operations and the extensive exercise of judgment by employees, must 
take adequate precautions to minimize the risk of employee misconduct. Our audit looked at various means that 
the Department employs to promote integrity among its employees. 


15.2 All of these means, taken together, form an important foundation for the promotion of integrity in 
Revenue Canada. The Department investigates incidents of misconduct; discipline is imposed on employees found 
guilty of misconduct; and, on the whole, management takes appropriate action, where weaknesses are highlighted 
by incidents, to prevent similar misconduct. Further, distributing statements of its mission, vision and values and 
three key booklets on conduct are important steps Revenue Canada has taken toward promoting an ethical culture. 
Most employees have received some formal training on expected conduct. 


15.3 There are areas where improvement will help Revenue Canada reinforce this sound foundation. Because 
the disciplinary process can be slow, its effectiveness as a deterrent to similar misconduct may be diminished. 
While management takes action to correct weaknesses highlighted by incidents, at times its responses are slow 
and focussed narrowly on local rather than department-wide weaknesses. Further, information provided on values 
and conduct is not presented in a way that allows easy understanding, and training is uneven. The Department 
takes few steps to monitor the level of employees’ understanding of expected values and conduct. Nor does it 
provide an individual whom employees can consult on a confidential basis on issues of integrity. 
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Management of the Social 
Insurance Number 


Chapter 16 - Main Points 


16.1 The Social Insurance Number (SIN) was implemented in 1964, to provide Unemployment Insurance, 
Canada Pension Plan and Quebec Pension Plan clients with a file number. In 1967, it also became a file identifier 
for Revenue Canada. From the outset, the SIN was the subject of intense parliamentary interest and debate. The 
controversy focussed largely on the implications of a potential expansion of its use to become a universal 
identifier. The history of the SIN has mainly reflected the tension between balancing the often competing 
objectives of maintaining individual privacy and improving administrative efficiency and effectiveness. 


16.2 Human Resources Development Canada has been administering the SIN in accordance with the intent of 
its legal framework whereby the SIN is intended to be a file identifier (account number) for certain federal 


government programs. However, the public often perceives the SIN to be a personal identifier or even an identity 
card. 


16.3 The use of the SIN outside the federal domain has evolved beyond the intent of the Treasury Board 
policy established in 1989, aimed at preventing it from becoming a national personal identifier. Our audit found 
that the SIN has become the common numerical identifier both inside and outside the federal government for a 
wide range of income-related transactions and benefits, among other uses. 


16.4 Changes to the Income Tax Act and Regulations have increased the use of the SIN at the 
provincial/territorial level and in the private sector, as have the growing number of data exchanges, comparisons 
and matches among various tiers of social programs. 


16.5 However, the SIN has a number of weaknesses. Some of them are well known. As the use of the SIN has 
become more widespread, these weaknesses have grown in importance. They include the following: 


e The information on SIN holders, particularly on births and deaths, is not always complete and 
accurate. 


e Existing SIN application procedures are insufficient to guard against fraud and abuse. Holders of 
11.8 million SINs who registered prior to the introduction of the proof-of-identity program in 1976 
have not been asked to provide proof-of-identity documentation. This exacerbates the risk of 
misrepresentation. 


e The provinces and financial institutions are required by the Income Tax Act and Regulations to 
collect SINs for tax purposes but cannot validate the SIN numbers provided by their clients. 


e SIN errors, abuse, and misuse affect many federal programs, provinces and the private sector. 
Collectively, the impact may be sizable. 


e Minimal effort is dedicated to investigations of SIN fraud and abuse, and penalties are minimal, with 
no real impact on deterrence. 

e Unregulated use in the private sector, except in Quebec, makes it vulnerable to abuse in terms of 
both privacy and fraud. 
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Chapter 16 — Main Points (cont'd) 


16.6 The effectiveness of certain administrative decisions as well as the integrity of social programs are 
greatly facilitated by the reliability of the information contained in the SIN data base. We believe it is time to 
review the current roles, objectives and uses of the SIN in light of its governmental and societal importance. The 
government needs to clearly state the level of integrity and privacy protection expected in the system. 


16.7 There appear to be two options: improve the existing framework to catch up with the current reality of 
SIN usage or else devise an acceptable alternative solution to meet the needs of users, including governments and 
individuals. In any case, the privacy implications need to be recognized. In our view, it is essential that Parliament 
play a major role in debating these issues, increasing public awareness, and finding a satisfactory solution. 
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Patented Medicine Prices 
Review Board 


Chapter 17 - Main Points 


17.1 The Patented Medicine Prices Review Board (PMPRB) is a small quasi-judicial board with an annual 
budget of approximately $3 million and 35 full-time-equivalent employees. Created in 1987, it has a mandate to 
protect consumer interests and contribute to Canadian health care by ensuring that prices charged by 
manufacturers of patented medicines are not excessive. However, the scope of its jurisdiction and the limitations 
of its consumer protection role are not widely understood. 


172 Since 1988, average annual increases in prices of patented medicines have moderated. The Board has 
contributed to the containment of patented drug prices, and has taken steps to assess the impact of federal price 
regulation. However, other factors also have contributed and we are concerned that the Board’s estimates of its 
savings to the Canadian health care system are overstated. In reporting its results to Parliament, the Board did not 
clearly identify the limitations of the estimates of its own impact. 


173 The audit identified issues pertaining to the legislative framework that need to be reviewed. The 
legislation is silent or unclear on certain areas. In addition, some requirements under the legislation are difficult to 
apply in practice, and consideration needs to be given to reviewing their continued relevance. 


17.4 The majority of patented drugs sold in Canada are priced within the Board’s guidelines. However, the 
Board needs to ensure that its price review decisions are clear and transparent. The Board also needs to identify 
cost-effective means to check the accuracy of price information submitted by manufacturers. Improvements are 
also required in the reporting of drug price trends and pharmaceutical research and development expenditures. 
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The Financial Information Strategy: 
A Key Ingredient in Getting 
Government Right 


Chapter 18 — Main Points 


18.1 Under the Financial Information Strategy (FIS), the government is making sweeping changes in the type 
of financial information provided to decision makers. FIS involves the move to full accrual accounting similar to 
that used by business firms, the implementation of new financial systems and the integration of full accrual 
financial information into day-to-day decision making of departmental managers. When fully implemented, by 
2001 according to the current plan, FIS should help the government strengthen significantly its management of 
business lines and its accountability to Parliament. The government has clearly indicated that it is committed to 
getting government right and has established FIS as one important initiative in doing this. The Office of the 
Auditor General fully supports the stated objectives of FIS. 


18.2 FIS began a decade ago. Progress in the early years was slow, but in recent years the tempo of work has 
picked up. To date, FIS has focussed on implementing new departmental financial systems that are Year 
2000-compliant and on ensuring that departmental and central systems will provide appropriate information to 
include in the government’s annual financial statements. 


18.3 However, the government has yet to implement its plans to deal with the important area of making full 
accrual financial information available to officials within departments and agencies who manage business lines 
and related components on a day-to-day basis. One of the objectives of FIS is to provide officials with more 
complete information on costs to compare with results achieved when making key decisions. Until this is done, 
the full benefits of FIS will not be realized. 


18.4 While better cost information for decision making is a benefit of FIS, we note that neither individual 
departments nor the government overall are aware of the full costs of FIS implementation, which could be 
significant. 


18.5 The government and Parliament can help ensure the success of FIS by appropriating funds on a full 
accrual basis, so that accountability and reporting are on the same basis. But central agencies and individual 
departments must also act to help ensure that the full benefits of FIS are achieved. Central agencies must secure 
complete departmental “buy-in”, and deputy ministers of individual departments must put in place the necessary 
infrastructure, including appropriate systems and human resources. 


18.6 Departments face a major challenge in completing the renewal of their financial systems to meet the 
objectives of FIS. Best practices and lessons learned from others who have gone before can help achieve success. 


18.7 Given the massive scale of financial systems renewal now under way to meet the Year 2000 challenge, 
we believe that there is a unique opportunity to put FIS in place by the 2001 target implementation date. This is 
not a simple task. To obtain the full benefits of FIS and thus have in place the information required for 
government today, the challenges faced by the government and summarized in this chapter must be met. In future 
audits, we will continue to track the government’s progress in implementing FIS and achieving these benefits. 
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Electronic Commerce 


Conducting Government Business 
via the Internet 


Chapter 19 — Main Points 


19.1 Research groups and experts in the information technology industry have predicted a phenomenal growth 
in the use of the Internet for business. In government, the Internet provides an opportunity to streamline operations 
and improve service delivery, as well as an alternative mode of conducting business. In a policy statement, the 
government committed to making electronic commerce its preferred way of doing business by 1998. In support of 
its agenda for “connecting Canadians”, it has also undertaken to become a model user of the information highway 
by 2000. 


19.2 The audit focussed on the government’s use of the Internet for internal and external purposes. We 
concluded that progress is being made in all three areas we examined, areas that are fundamental to support 
electronic commerce: 


e the government’s public key infrastructure project as a measure for secure electronic commerce; 
e the legal framework to support conducting business electronically; and 


¢ common technology infrastructures for interoperability and seamless access to information and 
services across departments and agencies. 


19.3 Nevertheless, we noted several key risks that could undermine the government’s public key infrastructure 
project. Progress in designing business processes to make use of the Internet and developing the related computer 
applications is lagging behind the progress of the infrastructure project. Further, the need to certify the public, a 
necessary element in delivering services externally, has yet to be addressed. 


19.4 A “paper bias” has been identified in the language of federal statutes and steps have been taken to 
resolve it. Further action is needed to manage the government’s potential legal liability in its use of electronic 
commerce. 


19.5 A senior sponsor is needed to advance electronic commerce in government. Many issues remain to be 
resolved before adequate common infrastructures exist to support the delivery of services across multiple 
departments and agencies. 


19.6 If the government does not address on a timely basis the identified risks and areas requiring action, its 
objective of making electronic commerce its preferred way of doing business may not be fully realized and its 
goal of becoming a model user of the information highway by 2000 may also be in jeopardy. 
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Preparedness for Year 2000 


Government-Wide Mission-Critical 
systems 


Chapter 20 - Main Points 


20.1 The Year 2000 problem was caused by the past programming practice of coding the year with two digits 
rather than four. On 1 January 2000, certain computer applications and devices could fail or malfunction. In 1997, 
we reported that the government’s rate of progress in preparing for Year 2000 had generally been slow; if it were 
to continue at the same rate, government systems probably would not be ready in time. 


20.2 In 1998, we audited some key computer systems and devices that support six mission-critical functions in 
government. The functions were inspection services to contribute to food safety; emergency assistance and 
support; income security services; First Nations transfer payments and trust funds; border crossing services for 
commercial goods; and law enforcement services. We concluded that as of 30 June 1998, several key systems 
supporting three of those functions remained at risk of not becoming fully compliant before 2000. 


20.3 We observed that the government has accelerated the pace of its efforts since our 1997 audit and has 
made progress in various areas, including: 


e ranking Year 2000 as a top priority; 


e¢ monitoring government-wide progress of systems and devices that support mission-critical functions 
in government; 


e advancing certain common horizontal initiatives; and 
e engaging the concern of ministers and advising them of the urgency of Year 2000 efforts. 


20.4 However, our review of the 30 June 1998 survey results summarized by the Treasury Board Secretariat 
showed “good” progress in only four departments, accounting for 7 out of 48 government-wide mission-critical 
functions. Nine departments and agencies responsible for 18 mission-critical functions had a score of 50 percent or 
less for completion of the work needed to make the systems supporting the functions compliant. 


20.5 We concluded that as of June 1998, various government systems supporting mission-critical functions 
remained at risk. With 18 months left, we are very concerned that some essential government services may be 
interrupted at the start of 2000. 


20.6 To ensure that mission-critical functions will continue, we recommend that the focus on Year 2000 be 
sustained, that the Treasury Board Secretariat consider strategic intervention as appropriate, and that contingency 
plans be prepared and tested for those systems that remain at risk. We also recommend that further action be taken 
to advance common horizontal initiatives and that reporting of Year 2000 information to Parliament be improved. 
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Canadian International 
Development Agency 


Geographic Programs 


Chapter 21 - Main Points 


21.1 In our 1993 Report, we raised a number of concerns about the performance of CIDA’s Bilateral Programs. 
Our 1995 and 1996 Reports contained a description of CIDA’s actions to implement an ambitious program of 
change and renewal. In this chapter we report on the extent to which CIDA has resolved the main concerns we 
raised in 1993. 


21.2 CIDA has adopted the concept of results-based management to change the Agency into a more 
results-oriented and learning organization. Progress is evident. CIDA has introduced a Results-Based Management 
Policy Statement, and supporting policies and guidelines have been developed and communicated to staff. It has 
established a Framework of Results and Key Success Factors as a guide for its managers and partners to focus 
more on results as they manage projects. However, progress in applying the principles of results-based 
management to CIDA contracts has been slow. 


21.3 CIDA is in the process of developing, reviewing and updating Regional/Country Development Policy 
Frameworks for all major countries and regions. Some of the more recently developed Frameworks state 
objectives and expected results. However, the expected results are not compared with actual results, one of the key 
means of assessing performance. 


21.4 At the project level, where official development assistance (ODA) is actually delivered, we found a 
greater emphasis on results, but uneven progress toward actually managing for results. Critical assumptions for 
project success and expected results have been articulated and a system of annual progress reporting implemented, 
but the statements of expected results are not always realistic. Further, once critical risks have been identified and 
assessed, they are not consistently monitored and addressed. This calls into question whether the expected benefits 
of projects will be realized. Projects are not systematically monitored after funding ends to determine whether the 
results expected have been achieved. 


21.5 CIDA prepares internal reports on performance, reflecting its increased ability to assess and report 
systematically on its performance. It also submits an annual Performance Report to Parliament; this would be 
improved if CIDA showed progress toward results for individual projects in the context of the results expected. In 
reviewing the Performance Report, we concluded that more rigour is needed to ensure that information is clear, 
meaningful, and accurate. The delivery of ODA in a great many countries and under uncertain political and 
economic conditions is a very complex and challenging undertaking, one that results in varying degrees of 
success. We believe it is important that CIDA bring out this dimension in its reporting; a more balanced picture 
would enhance the credibility of its reports. 


21.6 We concluded that CIDA’s actions have addressed the main concerns raised in 1993. The Agency now 
needs to keep up its momentum in implementing management for results. It has much of the supporting 
framework in place. To close the accountability loop, it needs to continue working on the measurement and 
reporting of development results. 
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The Federal Science and 
Technology Strategy 


A Review of Progress 


Chapter 22 — Main Points 


221 It has been nearly three years since the government released Science and Technology for the New Century 
—A Federal Strategy. The Strategy outlined three goals to which science and technology (S&T) resources were to 
be directed. It recognized the need for better management of S&T activities and provided a set of principles to 
improve their management in departments and agencies. 


22.2 In this follow-up to our 1994 audit of the federal S&T effort, we found that progress in establishing the 
elements required to improve its management has, overall, been slow. The government has not acted fully on 
some of the commitments it made in the Strategy. As a result, the management regime set up to oversee the 
federal S&T effort is not yet working as intended by the government — that is, ensuring that priorities are clear, 
that activities are co-ordinated, and that performance is reported on fully. 


22.3 In the departments we examined, progress in acting on the Strategy’s seven operating principles has 
varied considerably. However, even the less advanced departments have moved in the direction desired by the 
government. More remains to be done, and our follow-up identified three areas that require special attention: 
mission-driven, results-based research; scientific excellence; and the use of partnerships. 


22.4 Some results of the Strategy are now emerging, and we believe it may be time for the government to 
clearly set out what remains to be done — what is next on the agenda. Along with sufficient leadership and 
ongoing parliamentary oversight, this would help to ensure that the current Strategy does not become, like similar 
efforts over the last 30 years, one more missed opportunity. 


28 Report of the Auditor General of Canada — December 1998 


Veterans Affairs Canada 


Disability Pensions 


Chapter 23 — Main Points 


23.1 The disability pension program has annual expenditures of about $1.1 billion, making it the largest 
program of Veterans Affairs Canada. Disability pensions are paid to approximately 151,500 recipients — 68,000 
veterans, 25,500 former or current peacetime members of the Canadian Forces and 58,000 survivors. 


23.2 In September 1995, pension reform legislation came into effect. The legislation resulted in fundamental 
changes in responsibilities for the disability pension program. The Department took on new roles in providing 
assistance and counselling to disability pension applicants and in adjudicating first decisions for pension claims. 
The Veterans Review and Appeal Board was formed at that time to hear reviews and appeals of decisions. 


23:3 The main focus of the disability pension program since pension reform has been on reducing the time it 
takes to make pension decisions. Veterans Affairs Canada and the Veterans Review and Appeal Board have 
exceeded their targets for reducing turnaround times for disability pension decisions. 


23.4 The quality and consistency of the services provided in the preparation of first applications could be 
improved through greater use of standardized guidance materials and training for all those involved in the 
preparation of first applications. The table of disabilities, which is used in determining the nature and extent of 
disability for pension purposes, provides limited guidance for assessing the extent of some disabilities. To ensure 
consistent and high-quality disability pension decisions, the Department needs to improve its guidance materials 
that facilitate entitlement and assessment adjudications. 


23.5 Veterans Affairs Canada does not provide reasons for assessment decisions. There is a need for the 
Department and the Veterans Review and Appeal Board to regularly analyze the reasons for overturned pension 
decisions. 


23.6 Veterans Affairs Canada is not fully utilizing the departmental review process. This process was put in 
place by pension reform legislation and allows certain initial decisions to be reviewed more efficiently and in less 
time than a formal review request to the Veterans Review and Appeal Board. 


23.7 Within the next 10 years, the mix of disability pensioners will change considerably, with a significant 
increase in the number of former or current peacetime members of the Canadian Forces. The Department needs to 
inform Parliament of the changes in its client base and the strategy for addressing those changes. 


23.8 Veterans Affairs Canada has assigned a high priority to its Year 2000 projects. It has a plan and schedule 
that is expected to achieve Year 2000 compliance for its mission-critical systems by the end of 1998. However, the 
Department is also dependent on other systems for information, and disruption of those systems could affect the 
delivery of the disability pension program. 


Report of the Auditor General of Canada — December 1998 


2 


Revenue Canada 


International Tax Directorate 
Human Resource Management 


Chapter 24 - Main Points 


24.1 Revenue Canada’s International Tax Directorate was established in November 1991. It is the focal point 
for all international tax issues (except legislative issues), with a view to enabling Revenue Canada to respond to 
them in a consistent and timely way. 


24.2 A recent report of the Technical Committee on Business Taxation noted the following: 


e In 1993, taxpayers reported to Revenue Canada $248 billion in related-party cross-border 
transactions, $166 billion of which were between related parties in Canada and the United States. 


e In 1995, foreign direct investment into Canada was approximately $170 billion, or 20 percent of 
assets held by Canadian businesses; and outbound foreign direct investment by Canadians was about 
$140 billion, or 18 percent of total assets held by Canadian businesses. 


e Even relatively small shifts in income allocated to Canada from related-party transactions have the 
potential to cause a significant change in the domestic revenue base. 


24.3 In this complex and growing field, the Directorate requires staff who are highly skilled and who 
understand Canadian tax laws and tax laws in other jurisdictions. 


24.4 Weaknesses in human resource management in the Directorate, coupled with the often cumbersome 
human resource management rules in the public service, have resulted in long delays in the competition process 
for staffing positions and in finalizing appointments. Skilled candidates for international tax positions at senior 
levels are rare, and are in demand in the private sector. Good candidates may lose interest if they have to wait long 
periods before being offered a position. 


24.5 Currently, key positions in the Directorate — including positions of leadership — are filled by using 
secondments, redeployments and acting assignments. We are concerned that this approach to staffing may result in 
a lack of continuity and stability and may pose a risk to both the quality of work performed and the Directorate’s 
ability to fully realize potential revenues and protect Canada’s tax base. 


24.6 Despite having recognized since 1994 the urgent need for a human resource plan, the Directorate is still 
developing a comprehensive human resource plan and strategies linked to its business plan. During September and 
October 1998 it initiated a number of actions that, upon completion, it expects will address certain identified 
problems. 


24.7 Existing human resource and performance data bases need to be enhanced and better used to provide 
managers with information needed for human resource planning. 


24.8 If Parliament approves the establishment of the proposed new Canada Customs and Revenue Agency, the 
Department will become a separate employer and assume most of the responsibility for human resource 
management. In the absence of a comprehensive human resource plan and strategies linked to the Directorate’s 
business plan, the establishment of the new agency will not in itself resolve the problems outlined in this chapter. 
It is important that the analysis, planning and implementation of needed human resource initiatives be carried out 


as soon as possible to ensure that the Directorate can appropriately manage the risks to Canada’s tax base that are 
inherent in international transactions. 
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Transport Canada 


Investments in Highways 


Chapter 25 — Main Points 


25.1 During the last 10 years, Transport Canada has spent about $1.6 billion on provincial and territorial 
highways. At the time of our audit, it was involved in 24 multi-year cost-sharing agreements with provinces and 
territories under the Department’s various highway investment programs. Over the last five years, annual total 
expenditures under these programs have averaged more than $200 million. 


25.2 Each agreement under the highway investment programs is administered by a committee of one federal 
and one provincial official. Transport Canada has provided little or no support to guide and facilitate the work of 
the four departmental employees who represent the federal interest on these committees. 


25.3 Our analysis of Transport Canada’s performance in managing and administering its highway investment 
programs revealed a number of significant weaknesses. The programs themselves have been managed and 
administered more like grant programs than the contribution programs they are (the latter require that funding be 
conditional on performance and on compliance with provisions of an applicable agreement). We noted the lack of 
a number of sound financial management practices. We also found deficiencies in the information supplied to 
decision makers in support of proposed new programs. In particular, information on the condition of roads was not 
properly disclosed; potential environmental impacts of proposed highway programs at the strategic 
decision-making level were not assessed, as is required; and alternative financing arrangements by provinces were 
not identified, nor were their implications for traditional cost-sharing arrangements evaluated. 


25.4 Further, because its environmental screening and monitoring of projects lacked rigour and were poorly 
documented, we could not conclude whether Transport Canada had met its statutory environmental 
responsibilities when assessing (screening) individual projects. Moreover, we found a number of cases in which, 
contrary to statutory requirements, payments had been made before environmental screenings were completed. 


25.5 Despite the hundreds of millions of dollars it has spent on highway investments over the past 10 years, 
Transport Canada has yet to conduct a program evaluation of its highway investment programs. Accountability 
reporting to Parliament needs to be strengthened. 


25.6 Transport Canada is at a crossroads. There has been some pressure by the provinces for the federal 
government to renew or confirm its position on highway investments. The Minister of Transport has said that the 
existing national highway policy needs to be updated. Transport Canada needs to re-examine the National 
Highway Transportation Policy and make recommendations to the government as appropriate. 
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Contracting for 
Professional Services 


Selected Sole-Source Contracts 


Chapter 26 — Main Points 


26.1 To ensure that the principles of best value and open access are followed in contracting for goods and 
services, the government’s contracting regulations require that contracts valued at more than $25,000 be let, 
except in very limited circumstances, through a competitive, open tendering process. Recent work by our Office 
as well as a report by the Standing Committee on Government Operations show that departments use sole-source 
contracts extensively. This is a matter of considerable concern because sole-sourcing works against the principle 
of open access and puts the achievement of best value at risk. The total value of sole-source service contracts 
awarded in 1995 was about $1.4 billion. 


26.2 We audited a sample from five departments of 26 sole-source contracts for professional services. We 
selected this type of contract because previous work had led us to believe that it was a high-risk area. Most of the 
contracts in our sample would not, in our view, stand the test of public scrutiny. Nor do they reflect open access to 
contracting opportunities with the federal government. The total initial value of these contracts was $16 million. 


26.3 In our sample of contracts we found many instances in which needs analyses, definition of requirements 
and procurement planning had been inadequate. Departments often underestimate the nature and scope of the 
work required, and do not always clearly specify what contractors are to deliver. We could find little evidence of 
contract management: it often appears that the contractors determine what is required, how much effort it will take 
and at what point the job will be considered completed. 


26.4 The justifications for over two thirds of the sole-source contracts we audited did not conform to the 
exceptions allowed by the Government Contracts Regulations and as defined in the Treasury Board Secretariat’s 
Contracting Policy. This practice results in too many “one-horse races”. 


26.5 Departments issue many oral contracts that remain unsigned for long periods of time. Treasury Board 
states that the terms and conditions of any contract issued should be in writing and should be signed as soon as 
possible after the contract is awarded. 


26.6 In several cases departments were not able to demonstrate that their managers had, as required by the 
Treasury Board Secretariat’s Contracting Policy, documented and assessed the extent to which contractors had 
fulfilled the contract terms. In the absence of appropriate certification and documentation of what the contractors 


delivered, neither we nor management can be assured that due regard to economy was shown in the letting of 
those contracts. 


26.7 In most of the contracts we examined, government regulations were not followed. Clearly, we are 
concerned about this. In 1997, we examined the contracting process and concluded that the current regime that 
governs contracting was itself sound. Similarly, the Standing Committee on Government Operations examined the 
contracting process and concluded that the rules were sound. The problem we found is that departments disregard 
the rules that apply to sole-sourcing. Moreover, disregarding them appears to involve no significant consequences 


for either the managers responsible or their departments. (This was the first in a series of contracting compliance 
audits planned by the Office.) 
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Grants and Contributions 


selected Programs in Industry Canada 
and Department of Canadian Heritage 


Chapter 27 — Main Points 


27.1 The Office of the Auditor General considers the proper management of grants and contributions to be an 
important part of good public administration within the federal government. Accordingly, the chapter presents: 


e an overview of the use of grants and contributions by departments; 


e highlights of the Office’s observations over the past 21 years on the value for money in grant and 
contribution programs; 


e¢ aperformance management framework that we are developing for grant and contribution programs; 
and 


e the results of an audit, using aspects of this framework, of selected grant and contribution programs 
in Industry Canada and Canadian Heritage. 


27.2 Our audits of the management of grant and contribution programs over the past 21 years have produced a 
long series of consistent observations: problems in compliance with program authorities, weaknesses in program 
design, instances of poor controls, and insufficient performance measurement and reporting. Overall we have 
continued to find the same problems. There are many reasons why these problems have persisted. They range 
from decision-makers not following the rules governing expenditures on grants and contributions to weak 
management practices. To address this latter problem, our Office is developing a performance management 
framework. 


Ziles We found that there are significant opportunities to improve the management of the programs we audited: 


e For the Ontario Base Closure Adjustment Program delivered by Industry Canada, there was little 
evidence that the Department exercised due diligence in approving funding for many projects. 


e We had similar concerns about Canadian Heritage’s Multiculturalism Program. In about one third of 
the projects we audited, there was little evidence that the Department had exercised due diligence in 
approving funding. We are also concerned that the Program’s performance expectations are 
ambiguous; clarification is needed to avoid duplicating the efforts of other departments and of 
provincial governments. 


27.4 We also examined indirect delivery arrangements where a department contributes funds to an external 
organization that, in turn, distributes funding to the intended recipients. These arrangements require special 
accountability measures since departmental managers cannot rely on direct supervision and adjustment to achieve 
expected results. We found that Industry Canada could strengthen accountability for performance under its 
contribution agreements with CANARIE Inc. (Canadian Network for the Advancement of Research, Industry and 
Education) and PRECARN Associates Inc. (Pre-Competitive Advanced Research Network). 


Report of the Auditor General of Canada —- December 1998 


33 


Follow-up of Recommendations 
in Previous Reports 


Chapter 28 — Main Points 


28.1 The amount of progress toward meeting the recommendations of previous audits covers a broad 
spectrum. In some cases, all significant recommendations have been addressed. In other cases, there is little or no 
evidence of corrective action being taken. Most cases fall somewhere between these extremes. 


28.2 The following significant areas of concern remain: 


e Health Canada is not in a position to effectively co-ordinate and respond to a major nuclear accident 
affecting Canada. 


e Itis not clear whether the Code of Environmental Stewardship is still in place for those agencies not 
required to produce a sustainable development strategy and there is also a need to involve Crown 
corporations in the government’s greening efforts. 


e We observed that adequate strategic reconnaissance is still not being conducted prior to deployment 
of all peacekeeping missions. 


e Awareness of the potential risks to health, safety and the environment from federal contaminated 
sites, and the government’s lack of a comprehensive plan to manage its risks and associated costs is 
still an issue. 


e Information systems under development remain a concern due to their significance, in terms of the 
size of the investments and their impact on operations, and continuing delays in the planned 
implementation dates for certain systems. 


28.3 There are also successes to report: 


e Since our audit in 1996, materiel management has decreased reported inventories in four departments 
by more than $480 million. 


e Progress has been made in improving the quality of internal audit management. 


e Revenue Canada has taken steps to ensure that excise taxes and duties on selected commodities are 
correctly assessed, collected and reported. 


e Progress has been made in strengthening the control and accountability of the Canadian intelligence 
community. 


28.4 We are encouraged by the amount of progress reported, and urge departments and agencies to continue 
their efforts in addressing our concerns until all significant matters have been resolved through corrective action. 
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Other Audit Observations 


Chapter 29 — Main Points 


29.1 The Auditor General Act requires the Auditor General to include in his Reports matters of significance 
that, in his opinion, should be brought to the attention of the House of Commons. 


29.2 The “Other Audit Observations” chapter fulfils a special role in the Reports. Other chapters normally 
describe the findings of the comprehensive audits we perform in particular departments; or they report on audits 
and studies of issues that relate to operations of the government as a whole. This chapter reports on specific 
matters that have come to our attention during our financial and compliance audits of the Public Accounts of 
Canada, Crown corporations and other entities, or during our value-for-money audits. 


29.3 The chapter normally contains observations concerning departmental expenditures and/or revenues. The 
issues addressed generally involve failure to comply with authorities, and the expenditure of money without due 
regard to economy. 


29.4 Observations reported this year cover the following: 


¢ ongoing issues between Atomic Energy of Canada Limited and the government that need to be 
resolved; 


e _ transparency and the government’s Annual Financial Report; 
e concerns about poor control over computers loaned to employees for home use; 


e the urgent need for an updated long-term comprehensive plan to restore and renovate the 
Parliamentary Precinct; and 


e fairness in the Canadian income tax system. 


29.5 Although the individual audit observations report matters of significance, they should not be used as a 
basis for drawing conclusions about matters we did not examine. 
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If you wish to obtain other 1998 Report material, consult the Main Table of Contents and indicate the chapters and 
formats you wish to receive. Mail the form to: 


Office of the Auditor General of Canada 


240 Sparks Street, Stop 10-1 
Ottawa,ON KIA 0G6 
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Chapter 19 


Electronic Commerce 


Conducting Government Business 
via the Internet 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 


Table of Contents 


Main Points 


Introduction 


Electronic Commerce in Government 
A Predominant Area of Future Growth 
Focus of the Audit 


Observations and Recommendations 


A Public Key Infrastructure 
The government’s solution for secure electronic commerce 
Infrastructure development and management are under way 
Applications using public key infrastructure have yet to be identified and developed 
Need for further analysis of implications for dealing with the public 
Other development and implementation challenges to overcome 


Legal Issues 
Resolution of potential limitations in legal statutes is proceeding 
Legal liability is a concern that requires further analysis and resolution 


Action Needed to Develop Direction and Adequate Common Technology Infrastructures 
Senior sponsor needed to lead electronic commerce in government 
Further efforts required for technology standards and guidelines 


Electronic Initiatives in Government — A Profile 
Good practices and potential lessons learned 


Conclusion 


About the Audit 

Exhibits 

Ook Electronic Commerce: A Chronology of Major Events 

19.2 Glossary of Selected Terms Related to Public Key Infrastructure 


[Oe The Government of Canada Public Key Infrastructure Task Force and Committees 
19.4 Four Electronic Commerce Initiatives 


Page 


19-5 
19-7 


19-7 
19-9 
19-9 


19-10 


19-10 
19-10 
19-1] 
19-13 
19-14 
19-16 


19-18 
19-18 
19-19 
19-19 
19-19 
19-20 
19-22 
19-23 


19-25 


19-27 


19-8 
19-10 
19-12 
19-24 


Report of the Auditor General of Canada — December 1998 


19-3 


casi nun een aleians paler i emaqene evel Sree rs 3 
iiiormmtial yer ce i nt ba 
at tniren wsLaagy ¥ 


en a a | 
-aaltsitay hag tnalaett yyotiadical <i haven ates 
vifho 4 & —tarineseservi al auetalited lade 

frvrenal prmenest Se lsprnasing tens acateanny Barala, 
- wees onee . 
touh ab tad © | 

+ —_ a 

odd 


emnuil aS PMpighinnn®) A cepenqpead) wierweraalil) 12] 

ssadtactiead pie th Shall cel Geraint wenmate , O81 = 

Ride "pce det? =yes ii=—4 poh citar Bet . 
* heat sg ened”) (ahold gir. | SVE | 


Electronic Commerce 


Conducting Government Business 
via the Internet 


Main Points 


191 Research groups and experts in the information technology industry have predicted a phenomenal growth 
in the use of the Internet for business. In government, the Internet provides an opportunity to streamline operations 
and improve service delivery, as well as an alternative mode of conducting business. In a policy statement, the 
government committed to making electronic commerce its preferred way of doing business by 1998. In support of 
its agenda for “connecting Canadians”, it has also undertaken to become a model user of the information highway 
by 2000. 


19.2 The audit focussed on the government’s use of the Internet for internal and external purposes. We 
concluded that progress is being made in all three areas we examined, areas that are fundamental to support 
electronic commerce: 


e the government’s public key infrastructure project as a measure for secure electronic commerce; 
e the legal framework to support conducting business electronically; and 


e¢ common technology infrastructures for interoperability and seamless access to information and 
services across departments and agencies. 


193 Nevertheless, we noted several key risks that could undermine the government’s public key infrastructure 
project. Progress in designing business processes to make use of the Internet and developing the related computer 
applications is lagging behind the progress of the infrastructure project. Further, the need to certify the public, a 
necessary element in delivering services externally, has yet to be addressed. 


19.4 A “paper bias” has been identified in the language of federal statutes and steps have been taken to 
resolve it. Further action is needed to manage the government’s potential legal liability in its use of electronic 
commerce. 


19.5 A senior sponsor is needed to advance electronic commerce in government. Many issues remain to be 
resolved before adequate common infrastructures exist to support the delivery of services across multiple 
departments and agencies. 


19.6 If the government does not address on a timely basis the identified risks and areas requiring action, its 
objective of making electronic commerce its preferred way of doing business may not be fully realized and its 
goal of becoming a model user of the information highway by 2000 may also be in jeopardy. 
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Introduction 

Electronic Commerce in 
Government 

1937 In general, the term “electronic 


commerce” refers to business activities 
and transactions that are conducted 
electronically. It involves using 
information technology and 
telecommunications to disseminate 
information, exchange data and perform 
financial transactions. The term is not 
unique, nor is it defined specifically. Other 
terms that are used include E-commerce, 
E-business and E-trade. Electronic 
commerce encompasses an array of tools 
and technologies — such as electronic 
data interchange, electronic funds transfer 
and the Internet — that support many 
types of business applications. 


19.8 In its public consultation 
documents, the federal government has 
offered the following definition: 


Electronic commerce, which is at the 
heart of the information economy, is 
the conduct of commercial activities 
and transactions by means of 
computer-based information and 
communications technologies. It 
generally involves the processing and 
transmission of digitized information. 


19.9 For the government, electronic 
commerce involves using computers and 
telecommunications systems in a wide 
range of activities — from internal 
activities such as administration and 
operations to external activities such as 
providing services to the public. They can 
be financial, such as transferring funds, or 
non-financial, such as providing or 
exchanging data and information. 


19.10 Over the years, initiatives in this 
area (see Exhibit 19.1) have received the 
endorsement and support of 
interdepartmental senior management 
committees such as Treasury Board 
Secretariat’s advisory Information 
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Management Subcommittee (TIMS) and 
the Council for Administrative Renewal 
(CAR). 


19.11 In particular, the Treasury Board 
Secretariat identified electronic commerce 
as a key initiative in implementing the 
1994 Blueprint for Renewing Government 
Services Using Information Technology. In 
late 1995, the Secretariat received 
approval from Treasury Board ministers to 
implement the Blueprint. 


19.12. In December 1995, the 
Secretariat wrote to all deputy ministers to 
advise them that the Board had approved 
the Blueprint’s implementation and 
endorsed its specific measures and 
initiatives. Advancing electronic 
commerce was one of those initiatives. 
The Secretariat indicated the 
government’s intention to publicly 
announce that electronic commerce would 
be its preferred way of conducting 
business. Further, the Secretariat set the 
direction to achieve substantial progress in 
conducting internal activities 
electronically by 1997 and external 
activities by 1998. The letter also noted 
the government’s intent to become a 
model user of the information highway. 
On many occasions since then, the 
Secretariat has committed to making 
electronic commerce its preferred means 
of doing business with other governments, 
private sector organizations and Canadians 
by 1998. 


19.13. In the September 1997 Speech 
from the Throne, the government made 
the following commitment: 


The Treasury Board 
Secretariat has 
committed to making 
electronic commerce 
its preferred means of 
doing business with 
other governments, 
private sector 
organizations and 
Canadians by 1998. 


We will make the information and 
knowledge infrastructure accessible to 
all Canadians by the year 2000, 
thereby making Canada the most 
connected nation in the world. 


It went on to state that connecting 
Canadians would provide them with 
access to the skills and knowledge they 
need to benefit from the country’s rapidly 
changing knowledge and information 
infrastructure. A connected nation is 
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By “putting the 
government on-line”, 
the Canadian federal 
government would 
become a world leader 
in Internet-based 
delivery of government 
information and 
services. 


Exhibit 19.1 


Electronic Commerce: 
A Chronology of Major Events 


viewed as an important element in 
enabling Canadians to participate and 
succeed in a global, knowledge-based 
economy. 


19.14 The Electronic Commerce Task 
Force at Industry Canada has been leading 
the move toward “connecting Canadians”. 
In early 1998, it formulated a two-part 
strategy — domestic and international — 
to make Canada a world leader in 
electronic commerce by 2000. 


19.15 
on: 


The domestic agenda focussed 


e building trust in electronic 
commerce among businesses and 
consumers; 


e setting ground rules for the digital 
marketplace as necessary; and 


e connecting Canadians to the digital 
economy by improving their access to 
networks, enhancing their awareness and 
developing their skills. 


19.16 The international agenda dealt 
with the need for a global framework to 


1994 1995 


facilitate the international growth of 
electronic commerce. This would include 
a legal framework, rules to govern the 
marketplace and the development of 
infrastructure. An immediate priority for 
this part of the agenda was to support the 
October 1998 OECD ministerial 
conference, “A Borderless World — 
Realizing the Potential of Global 
Electronic Commerce”. Canada hosted the 
conference in Ottawa. 


19.17 The domestic agenda further 
outlined a role for the government as a 
model user of electronic commerce. By 
“putting the government on-line’, the 
Canadian federal government would 
become a world leader in Internet-based 
delivery of government information and 
services. This objective echoes a 1995 
recommendation by the Information 
Highway Advisory Council. It is also 
consistent with the direction set by the 
Treasury Board Secretariat in late 1995 to 
make electronic commerce the preferred 
way of doing government business by 
1998. 


1997 1998 


e Government issues 
technology document 
entitled Blueprint for 
Renewing Government 
Services Using 
Information 
Technology. 


e Information Highway 
Advisory Council 
publishes its first report 
containing 
recommendations on the 
government’s role as a 
model user of the. 
information highway. 


e Government commits to 
making electronic 
commerce its preferred 


way of doing business by 


1998. 


e Strategy paper entitled 
Electronic Commerce 


e Throne Speech 
commits the 
government to 
making Canada the 
most connected 
nation in the world 
by 2000. 


e Industry Canada 
establishes the 
Electronic Commerce 
Task Force to 
co-ordinate the 
development of a 
national strategy for 
electronic commerce. 


e Government reaffirms 
its commitment to 
become a model user 
of the information 
highway by 2000. 


Commitment and Agenda 
for Action provides a status 
report on what needs to be 
done government- wide to 
do business electronically. 
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A Predominant Area of Future 
Growth 


19.18 The government has been using 
many established technologies to replace 
paper-intensive and cumbersome 
processes, in administration and 
operations and in the delivery of programs 
and services. For example, it makes 
payments to millions of individuals 
through electronic funds transfer to their 
bank accounts, thereby eliminating the 
need to issue and process paper cheques. 
Taxpayers can file their income tax returns 
through electronic data interchange, 
replacing paper returns and related 
documents. Using kiosks and service 
centres, government provides key 
information to its clients electronically, 
on-site and without intervention from 
staff. By employing the X400 technical 
standard, government employees can 
exchange electronic messages (e-mail) 
across all departments and agencies. 
Today, e-mail has become an essential 
alternative to voice messages and paper 
mail as a means of communication. 


19.19 General literature has shown that 
the Internet is a predominant area of 
future growth in electronic commerce. 
While statistics on Internet use are 
modest, the growth to date has been 
impressive and the potential for future 
growth is tremendous. In the United 
States, business-to-business Internet 
commerce grew tenfold from 1996 to 
1997, when it reached US$8 billion; a 
research group has predicted that it will 
grow to $327 billion by 2001, a fortyfold 
increase. Another group has estimated that 
the number of U.S. companies using 
Internet commerce will increase from 
100,000 in 1998 to 600,000 by the year 
2000. A 1997 report by the Organization 
for Economic Co-operation and 
Development (OECD) noted that the most 
conservative private sector estimates 
predict a tenfold growth by 2000 in the 
volume of electronic commerce transacted 
over global information networks. 
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Business-to-business Internet commerce is 
expected to be a major force in North 
America and Europe over the next two 
years. 


19.20 By serving as a model user of the 
information highway and pursuing a 
domestic agenda of “connecting 
Canadians”, the government is helping to 
position Canada in a global, 
knowledge-based economy. Conducting 
business via the Internet is a key element 
in the government’s policy to make 
electronic commerce its preferred way of 
doing business. It provides an opportunity 
for new and innovative ways of 
interacting with the private sector, other 
levels of government and the public in 
delivering programs and services. It will 
help the government to meet anticipated 
demands from external users as access to 
and use of the Internet grow. Internally, 
electronic commerce can benefit 
government administration and operations 
by increasing efficiency and reducing 


General literature has 
shown that the Internet 
is a predominant area 
of future growth in 

electronic commerce. 


costs. 
Focus of the Audit 
19.21 The audit focussed on the 


government’s use of the Internet to 
conduct business — both internal and 
external — as part of its stated policy on 
electronic commerce and its commitment 
to serve as a model user of the information 
highway. 


19.22 The audit assessed the 
government’s progress in addressing three 
key areas that will enable it to do business 
over the Internet: a public key 
infrastructure as a measure for secure 
electronic commerce; a legal framework 
to support electronic commerce; and 
common technology infrastructures. In 
addition, we reviewed four government 
initiatives involving electronic service 
delivery, in order to identify good 
practices and possible lessons learned. 


19.23. Further information about the 
audit objective, scope and criteria can be 
found at the end of the chapter in About 
the Audit. 
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internet users require 
trust and confidence 
that transactions are 
safe and secure and 
that their concerns 
about privacy are 
adequately addressed. 


Observations and 
Recommendations 


A Public Key Infrastructure 


The government’s solution for secure 
electronic commerce 


19.24 In moving from traditional 
paper-based transactions to electronic 
commerce via the Internet, users require 
trust and confidence that the transactions 
are safe and secure and that their concerns 
about privacy are adequately addressed. 
New technical tools and security measures 


need to be developed to support a secure 
environment for conducting business 
electronically. 


19.25 In lieu of the traditional controls 
that provide assurance to users, different 
security measures are needed to keep 
electronic transactions safe. Cryptography 
provides protection in four areas, ensuring 
that business transactions are kept 
confidential (confidentiality), are not 
tampered with (integrity) and originate 
from the bona fide source that is claimed 
(authentication), and that their occurrence 
cannot be denied by the originator 
(non-repudiation). Exhibit 19.2 provides a 


Exhibit 19.2 


Authentication An assurance of identification that users are in fact who they claim 
to be, thereby guaranteeing that the message is legitimate. 
Glossary of Selected Terms | Pe 3 5 ry 
Related to Public Ke | Certificate Polic A general statement of policy governing the operations of a 
y: g poucy g g P 
u certification authority in managing its public keys and digita 
Infrastructure | horit ging its public keys and digital | 
certificates. 
Certification Authority | An authority trusted by one or more users to issue and manage 


digital certificates. 


Certification Practice 
Statement 


A detailed statement of the operational procedures, standards and 
practices used by a certification authority in carrying out its | 
functions under the certificate policy. 


Confidentiality A characteristic of information that is accessed only by authorized 


parties and no one else. 


| Cross Certification A process that allows two separate certification authorities to agree 
to respect each other’s certificates and keys so that their users can 
| Interact seamlessly with each other as if only one certification 


authority structure existed. 


Digital Certificate The equivalent of an electronic passport that holds the public key 
of the owner, together with basic information on the identity of 
that owner and, optionally, information on specific authority that 
may be exercised by the owner, as certified by a trusted 


certification authority. 


A characteristic of information that has not been altered in an 
unauthorized manner. 


Integrity 


Based on evidence that the information has in fact been sent and 
received, the sender cannot successfully deny having sent it or the 
| receiver having received it. | 


Non-repudiation 


Root Authority A trusted primary certification authority that certifies bodies under 
it as accredited certification authorities. It implements the 
approved certificate policy and provides a common point of 


reference for cross certification. 


Source : Adapted from Treasury 
Board Secretariat documents 
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glossary of selected terms related to public 
key cryptography services. Those terms 
appear in italics when they are first used 
in the chapter. 


19.26 In 1995, the government 
identified the need for secure and private 
transmission of data as a key issue that 
needed to be addressed. Under the 
direction of a subcommittee of the 
Council for Administrative Renewal, work 
was begun to establish a public key 
infrastructure for the government. The 
public key cryptography techniques use 
technology to encrypt, decrypt and verify 
data. In addition to the technical 
component, an infrastructure is needed to 
manage and co-ordinate the administration 
in departments and agencies of security 
regimes that relate to public key 
cryptography services. 


Infrastructure development and 
management are under way 


19.27 Under the direction of the 
Council for Administrative Renewal and 
its subcommittee, an interdepartmental 
working group commissioned a business 
case for developing a public key 
infrastructure for government. The 
business case put forward three options for 
deploying public key techniques, and 
recommended that the government adopt a 
government-wide solution. 


19.28 The recommendation was 
accepted by the government. In late 1995, 
the Treasury Board endorsed the 
Government of Canada Public Key 
Infrastructure project (GOC PKI) and 
approved the related funding requirement. 
It was anticipated that the project would 
be delivered during 1998 and be ready for 
departmental use by December 1998. The 
Secretariat, as an oversight body, 
established and chaired a Policy 
Management Authority committee to set 
GOC PKI policies and provide a 
management framework for departments 
and agencies participating in the project. 
Working in partnership with a private 
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sector firm, the government’s 
Communications Security Establishment 
(CSE) played the role of project manager 
in the development of the technical 
product. The CSE was also assigned the 
role of root authority, a central facility for 
the government. 


19.29 In July 1996, the government 
entered into a contract with a private 
sector firm to develop a GOC PKI product 
to meet its requirements. In June 1998, the 
contractor released an interim GOC PKI 
product. The Communications Security 
Establishment expected that by December 
this release would be accepted on an 
interim basis and sanctioned for use by 
departments and agencies. 


19.30 Four of the six departments that 
participated in the initial funding of the 
project have been using early releases of a 
commercial version of the product on a 
pilot basis. They expect to upgrade to the 
interim GOC PKI product when the CSE 
provides interim approval. In addition, a 
number of other departments and agencies 
have expressed interest; some have 
acquired the commercial product on a 
limited basis and are using it in a test 
mode, primarily for internal secure 
messaging. 


19.31 The Policy Management 
Authority (PMA) committee established 
for the GOC PKI in early 1996 was 
chaired by the Acting Chief Information 
Officer of the Treasury Board Secretariat. 
A new Chief Information Officer was 
appointed in March 1997 and assumed the 
role of Chair in December 1997. Although 
Year 2000, the two-digit year code 
problem, became a top priority of the 
Chief Information Officer Branch, the 
GOC PKI project has remained a strategic 
priority. 

19.32 Inearly 1998, it was recognized 
that additional resources would be needed 
to advance the development of policies 
and the management framework for 

GOC PKI. The Secretariat established an 
Interdepartmental PKI Task Force that, as 
of July, had a staff of about 15. The Task 


via the Internet 


In late 1995, the 
Treasury Board 
endorsed the 
Government of Canada 
Public Key 
Infrastructure project. 
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Exhibit 19.3 


The Government of Canada Public 


Force director became Co-Chair of the 
PMA committee. By early July 1998, 
some 1] subcommittees and working 
groups had been set up, involving over 
300 staff from various departments and 
agencies (Exhibit 19.3). In addition, the 
Senior Interdepartmental Lead Committee 
of senior government managers was struck 
and took on an advisory role to the PMA 
committee. 


19.33, When we completed our audit in 
early July 1998 the government, under the 
direction of the PMA committee, had 
defined the need for eight certificate 
policies for GOC PKI. The policies 
represent four levels of security assurance: 
rudimentary, basic, medium and high. The 
PMA committee also directed the 


Key 


Infrastructure Task Force and Committees 


Communications 
Security Establishment 


development of, and subsequently 
approved, a framework of elements that 
need to be addressed in a certificate policy 
and a certification practice statement. 


19.34 The certificate policy and 
certification practice statements are 
necessary instruments to govern the 
implementation and management of a 
public key infrastructure. These 
instruments are essential in cross 
certification, a process designed to ensure 
trust and confidence among users in 
different organizations that use a public 
key infrastructure. The Interdepartmental 
PKI Task Force was preparing draft 
policies and it expected the drafts to be 
complete and ready for consultation by 
August 1998. When the certificate policies 


S C11 OL a eee as ee 
Interdepartmental 
Lead Committee 


| Treasury Board Secretariat 


Interdepartmental GOC 


PKI Policy Management 


Steering Project Office Authority 


Committee 


Source: Treasury Board Secretariat 


Interdepartmental PKI 
Task Force 


Policy and Legal 
Subcommittee 


Working Groups 


Advanced Card Technology 


Directory 


Electronic Commerce 


ee pene or Intergovernmental Relations 
and Interoperability Establishment and 
Subcommittee Documentation 


Subcommittee 
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Electronic Documents 


Procurement Client 
Advisory 


Secure Message Handling 
System 


Report of the Auditor General of Canada — December 1998 


Electronic Commerce: Conducting Government Business 
via the Internet 


are adopted, all departments and agencies 
choosing to participate in the GOC PKI 
will develop certification practice 
statements that are in accordance with 
them. 


19.35 We found, as of July 1998, that 
progress had been made in various areas 
of the GOC PKI project. An interim 
technical product had been released and 
was awaiting approval by the 
Communications Security Establishment; 
the management framework for the 
infrastructure component was being 
developed. However, we also observed 
several major risks to the project, which 
the following paragraphs discuss. 


Applications using public key 
infrastructure have yet to be identified 
and developed 


19.36 The business case for the 

GOC PKI project was prepared in order to 
identify options and recommend a course 
of action for providing public key 
cryptography as a measure for secure 
electronic commerce in government. 
Consequently, the options revolve around 
alternative ways of providing public key 
services. The analysis supporting the 
business case contemplated future 
requirements for public key cryptography 
services. However, the project has so far 
had limited involvement in identifying 
and facilitating the development of 
electronic business applications that 
would use the GOC PKI product when it 
becomes ready for use. 


19.37 In preparing the business case, a 
working group conducted a survey to 
determine the extent of the demand in 
government for a public key infrastructure 
to support electronic commerce. The 
survey sampled 26 departments, agencies 
and Crown corporations, representing 
about 15 percent of government entities 
and some 60 percent of federal 
employees. In addition to the Treasury 
Board Secretariat about 14 entities, 
primarily larger departments, responded to 


the survey. Although the survey had 
attempted to directly involve government 
business managers and strategic business 
planning personnel, the survey report 
observed that responses from some 
departments had been prepared by 
technical specialists in information 
security. While the report noted that those 
specialists were familiar with their 
departments’ strategic business objectives, 
it also acknowledged that this factor could 
have affected the quality of the assessment 
of demand. 


We found, as of July 
1998, that progress 
had been made in 
various areas of the 
government’s Public 
Key Infrastructure 
project. 


19.38 The survey report summarized 
the projected demand for encryption and 
electronic authorization services from 
1995 to 1999 by user type, such as 
government employee, private sector 
company and citizen. The requirements 
were also categorized by degree of 
certainty, from possible to likely to firm 
requirement. 


19.39 The results showed that by 1999 
there would be firm requirements for 
about 45,000 government employee users 
and 8,000 private sector company users of 
encryption and electronic authorization 
services; no firm requirements among 
citizen users were projected. By 1999, 
however, it was estimated that government 
employee users could exceed 110,000 and 
citizen users could reach 5.5 million. The 
cost projections in the business case were 
based on information from the survey 
report. 


19.40 During 1998, the Secretariat 
convened several meetings to discuss the 
demand for the commercial product in 
order to plan a government-wide 
procurement strategy. Many departments 
and agencies expressed interest, but most 
were not ready to commit to a purchase; 
the product’s lack of business applications 
was identified as a major barrier. A 
common application that has emerged as a 
driving force for using the commercial 
product would be internal secure 
messaging among employees. This is 
consistent with the projections made in the 
survey report. It estimated that of the 
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Action is needed to 
encourage 
government program 
managers and 
business planners to 
consider using the 
Internet as an 
alternative means of 
delivering services. 


45,000 government employee users 
expected to require the services by 1999, 
some 25,000 would require them for 
internal secure electronic mail. 


19.41 Inthe meantime, program 
managers and administrators in 
departments and agencies have been 
considering alternative means of 
delivering services and streamlining 
administrative and operational processes, 
mostly independent of the GOC PKI 
project. Through various interdepart- 
mental committees and working groups, 
as well as senior management advisory 
and oversight committees, there is a 
general awareness and exchange of 
information on different business 
applications and initiatives being 
developed. Yet no direct action has been 
taken to co-ordinate new business 
processes that use the Internet with the 
development of corresponding computer 
applications and the public key 
cryptography services that support them. 


19.42 There is a risk that the GOC PKI 
project will not be used to its full potential 
upon delivery. In our view, without direct 
action it is unlikely that business 
processes using the Internet and the 
corresponding computer applications will 
be ready when the public key 
cryptography services provided through 
this project become available and are 
officially approved for use in government. 
While the Year 2000 date code problem 
takes priority, action is needed to 
encourage government program managers 
and business planners to consider using 
the Internet as an alternative means of 
delivering services. Further, there is a risk 
that applications may focus on internal 
administrative and operational processes, 
although the government’s technology 
Blueprint also envisioned external 
electronic service delivery. 


Need for further analysis of 
implications for dealing with the public 


19.43 Users of public key cryptography 
services need to be registered before they 
can be recognized as users. They are 
registered only after their identity, level of 
authority and required level of assurance 
are validated. Registered users are 
assigned unique digital certificates. The 
body that issues and manages these 
certificates is known as a certification 
authority. Like electronic passports, the 
certificates identify the specific registrants 
and their level of authority and assurance. 
They support the processes of encrypting, 
decrypting and verifying the transmitted 
data. Users under the same certification 
authority should be in a position to enjoy 
the benefit of secure transactions among 
themselves. 


19.44 In anera of open public 
networks, users under different 
certification authorities will need to 
interact with one another, and certain 
transactions will require a secure 
environment. Confidence and trust in the 
system can be maintained only if 
certification authorities trust each other. 
This trust can be built through cross 
certification, a process by which the 
security regime and the administration of 
digital certificates are verified. Cross 
certification also involves matching 
assurance levels for transactions. 


19.45 The process of cross certification 
is technical and complex and poses 
management challenges among different 
government organizations. It is a prime 
reason for the government to develop 
strict certificate policies and the 
infrastructure component of its GOC PKI 
project. The extension of public key 
cryptography services beyond government 
into the public domain further complicates 
the process. It also introduces many other 
issues. 
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19.46 For example, a government 
employee would logically be registered by 
either his or her own organization or 
another government entity charged with 
that responsibility. For members of the 
public, however, it is not obvious who the 
certification authority should be, nor is it a 
trivial matter. 


19.47 ‘If the government were to take on 
the responsibility of certifying members of 
the public, it could have each department 
and agency register public users of its own 
programs and services, or set up a single 
point of contact to register public users for 
all their transactions with different 
departments and agencies, or some 
combination of the two. Alternatively, the 
government could have public users 
register with private sector certification 
authorities and it could cross certify with 
those authorities to support secure 
electronic commerce transactions between 
the government and the public. 


19.48 Each option has its advantages, 
disadvantages and ramifications. An 
every-department-for-itself approach 
would cause duplication and result in 
higher costs. In addition, members of the 
public might have to possess many digital 
certificates in order to receive services 
from the government or transact with it. 
At issue is the notion of “one user, one 
certificate” — that is, a public user would 
obtain and use a single set of certificates 
in doing business with the government. 
Whatever the government’s eventual 
policy on issuing certificates, some 
members of the public would expect the 
government to make every effort to 
minimize the number of certificates they 
need. 


19.49 In comparison, having a single 
government organization serve as the 
certification authority for public users 
could raise privacy issues and exposure to 
legal liability for the Crown. It would 
necessitate maintaining a high 
concentration of personal information 
electronically on one site, introducing the 
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risk that the government certification of a 
public user could be misused or abused. 


19.50 If the government were to use 
private sector certification authorities, 
public users would register with them and 
the government would need to cross 
certify with each of those authorities. This 
would be more complex and could pose a 
significant challenge. In addition, the 
government would need to address policy 
issues such as whether to select private 
sector certification authorities for cross 
certification, whether to limit their 
number, and whether and to what extent 
the government ought to encourage 
private sector investment in this area. 


19.51 Another issue is the potential cost 
to the public. Regardless of the 
certification authority chosen, costs will 
be incurred for the initial registration of a 
public user, for ongoing maintenance, and 
for changes and renewals as necessary. If 
it were a government service, the extent to 
which costs would be recovered through 
user fees would be a policy matter; if it 
were a private sector service, the public 
would likely be charged a fee. These 
factors will impact the services that the 
government wishes to provide 
electronically and it will need to consider 
them carefully. 


Some members of the 
public would expect 
the government to 
make every effort to 
minimize the number 
of certificates they 


19.52 While the demand survey need. 


supporting the GOC PKI project forecast a 
significant requirement for public use of 
encryption and authorization services, its 
technical architecture focussed primarily 
on departments and agencies. There was 
limited dialogue between the Policy 
Management Authority committee and 
potential private sector service providers 
before 1998. It was only in recent months 
that the implications of certifying the 
public began to be analyzed. A May 1998 
report of one interdepartmental working 
group recommended that resources be 
allocated to conduct an in-depth 
assessment and feasibility analysis and to 
identify a preferred option for registering 
external users. 
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The issue of certifying 
public users is 
complex and has 
major implications. 


Public key cross 
certification is new 
and complex, and it 
has yet to be 
demonstrated in live 
applications. 


19.53 The issue of certifying public 
users is complex and has major 
implications. If the government does not 
address it in a timely manner, other 
options may emerge and later complicate 
the task of co-ordinating a common 
approach. This could significantly 
compromise the potential return on the 
government’s investment in its GOC PKI 
project and curtail the scope of secure 
government business conducted via the 
Internet. 


Other development and implementation 
challenges to overcome 


19.54 Public key cross certification is 
new and complex, and it has yet to be 
demonstrated in live applications. It calls 
for regimented discipline and strict 
adherence to certificate policies and the 
related practice statements. Within 
government, the current technical 
architecture contemplates seven 
certification authorities at the 
departmental level, with the 
Communications Security Establishment 
(CSE) serving as the central facility to 
conduct cross certification. Draft 
certificate policies are still being 
developed and have yet to be adopted by 
the certification authorities. The 
application of cross certification could 
pose a major challenge to implementing 
the government’s public key infrastructure 
project. 


19.55 We also noted that product 
development is experiencing delay. The 
original contract called for the technical 
product to be delivered in 1998, in time 
for the government to accept it and 
approve its deployment by the end of the 
year. 


19.56 The contractor delivered an 
interim version in June. Information 
provided by the Communications Security 
Establishment showed that an update will 
be provided in the fall, and interim 
approval for use is expected by December 
1998. According to the CSE, the interim 


product is being tested to support the 
rudimentary and basic levels of assurance, 
and possibly the medium assurance level. 
A final product supporting all levels of 
assurance is now expected for the summer 
of 1999, and the government’s final 
acceptance and approval by the end of that 
year. 


19.57 In addition, standards for 
cryptography products are evolving. 
Through the GOC PKI project, the 
government has adopted certain technical 
standards to allow different technologies 
in government to operate with one 
another. These standards are not 
sanctioned by a formally accredited 
standard-setting organization and they are 
subject to market forces. Since the 
standards are evolving, there 1s a risk that 
they may be superseded by others, causing 
further delays in product development and 
possibly leading to incompatibility with 
some trading partners and parts of the 
private sector. 


19.58 The business case for the project 
estimated a total cost, for 16 departments, 
of approximately $35 million in the five 
years ending 1999, and ongoing operating 
costs of about $4 million a year thereafter. 
The cost estimates did not allow for 
potential costs of technological upgrades, 
maintenance and support. In a rapidly 
changing field such as this, the 
understatement of annual operating costs 
could be significant. Furthermore, the 
business case focussed primarily on using 
cryptography within the government. The 
estimates did not include costs related to 
the certification of public users and the 
possible need to fund private sector 
certification authorities or to share costs 
with them. If the estimates do not reflect 
the full costs of a project, it will be 
difficult to assess and report performance 
at a later date. Moreover, it can call into 
question the project’s expected return on 
investment. 


19.59 The risks and challenges that we 
have noted can severely compromise the 
scope and use of the public key 
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infrastructure to provide a secure 
environment for electronic commerce. 
They can undermine the government’s 
policy of making electronic commerce its 
preferred way of doing business, as well 
as its goal of becoming a model user of 
the information highway. 


19.60 The government should act 
expeditiously to identify and develop 
applications requiring secure data 
communication. It should involve 
business and program managers from 
departments and agencies directly with 
the Treasury Board Secretariat’s Public 
Key Infrastructure Task Force and the 
Government of Canada Public Key 
Infrastructure project, to optimize the 
use of the infrastructure in support of 
secure electronic commerce throughout 
the government. 


19.61 The government should address 
concurrently the issue of certifying 
public users and the development of the 
technical product and management 
framework for its public key 
infrastructure, including consulting 
with the private sector and potential 
certification authorities for the public. 


19.62 The government should manage 
the project risks that have been 
identified, and others as they arise. In 
particular, it should: 


e implement some small applications 
as early as possible to demonstrate that 
cross certification among a small 
number of government certification 
authorities is viable and that different 
platforms of technology can 
interoperate; 


e maintain a close watch over 
evolving standards to ensure that its 
public key infrastructure remains 
compatible; and 


¢ broaden the cost estimates in the 
business case to better evaluate the 
return on investment and provide an 
appropriate basis for making decisions 
and reporting performance. 
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Government’s response: We are pleased 
that the Auditor General has undertaken 
an audit of electronic commerce. 
Electronic commerce or electronic service 
delivery will be one of the main vehicles 
used to improve services to Canadians. 
The rapid technological advances 
occurring in this field make this an area 
that is in constant change. Since the 
completion of the audit in July, progress 
continues to be made on a number of 
fronts. 


We agree with the need to involve business 
and program managers with the Treasury 
Board Secretariat's Public Key 
Infrastructure (PKI), and consultations 
with these groups are under way. A 
recently completed inventory of PKI 
activity across government lists over a 
hundred initiatives planned and/or under 
way. This consultation and information 
sharing will continue. A senior-level 
committee composed of business managers 
from across government is providing the 
program managers with leadership in this 
field. 


Consultations with the provinces and with 
the private sector on issues such as the 
certification of public users and cross 
certification are under way. The Public 
Sector Chief Information Officer Council 
created last March is examining these 
issues in the context of a harmonized 
approach to PKI implementation across 
Canada. The proposed model GOC PKI 
certificate policy is currently under review 
by both public and private sector 
organizations. 


The original implementation of PKI in 
seven departments to meet their internal 
needs was established as a formal project 
and is being managed as such by 
Communications Security Establishment. 
The government's move to electronic 
service delivery will consist of many 
projects initiated and managed by 
individual departments. The Treasury 
Board Secretariat will provide 
co-ordination and oversight to these 
initiatives collectively but will be careful 
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The Department of 
Justice conducted a 
review of federal 
statutes for language 
that might indicate a 
paper bias. 


not to compromise the accountability of 
the individual project managers. 


The overall approach to the public key 
infrastructure project is to use off-the-shelf 
commercial software. This is cost-effective 
and ensures more compatibility than a 
made-to-order approach. The Task Force 
is keeping a close watch over evolving 
standards to ensure continued 
compatibility. Consultations with other 
bodies across the country and 
internationally are providing the 
government with up-to-date information 
on the current trends and activities in this 
field. 


In developing their business cases, 
departments will not be making a case for 
PKI per se. They will be making business 
cases for PKI-enabled applications to 
meet specific business requirements. These 
PKI-enabled applications will permit 
departments to do business securely within 
the government, between governments and 
with Canadians. The government's 
approach is to complete Phase I, putting 
in place the infrastructure to support these 
PKI-enabled applications as they are 
developed and deployed. 


Legal Issues 


Resolution of potential limitations in 
legal statutes is proceeding 


19.63 In 1995, a working group was 
tasked by the Council for Administrative 
Renewal to identify and address legal 
issues associated with the government’s 
security strategy for electronic 
information. Its report provided an 
overview of the various legal issues 
involved. In March 1996, the Department 
of Justice mandated an Electronic 
Commerce Secretariat to provide legal 
advice on electronic commerce 
technology. It was recognized that there 
was no separate legal framework for 
electronic commerce and that legal and 
commercial frameworks designed for a 
non-digital age might need to be adapted 


for doing business in an electronic 
environment. A proposed framework 
included three objectives: 


e making statutes media-neutral 
(applicable to any medium); 


e ensuring the recognition of secured 
electronic authorization; and 


e revising evidence rules for electronic 
records. 


19.64 The Department conducted a 
review of federal statutes for language that 
might indicate a paper bias. Through an 
electronic search of key words such as 
“forms”, “notarized” and “writing”, the 
Department found that 330 of the nearly 
600 statutes contain provisions that use 
such terms. 


19.65 Further to the review, instead of 
amending the statutes individually the 
Department’s assessment was that 
preparing general or global provisions 
would be more efficient and provide for 
more consistency. Among other things, the 
general provisions would serve as a tool 
for interpreting existing statutes in a 
media-neutral way. They would represent 
an additional option and would not 
preclude the use of paper-based 
mechanisms under existing statutes. 


19.66 Moreover, the Department of 
Justice determined that the Canada 
Evidence Act ought to be amended to 
address the introduction of electronic 
records into evidence. The 1995 working 
group had also identified a possible need 
to review and amend this Act. 


19.67 Inearly May 1998, the 
Department released a consultation paper 
entitled “Facilitating Electronic 
Commerce: Statutes, Signature and 
Evidence”. The paper put forward the 
Department’s proposal, as described in 
paragraphs 19.65 and 19.66, to resolve the 
potential limitations that it had identified. 
It expected to conclude consultations over 
the summer. Subsequent to our audit, the 
Department advised us that it had received 
endorsement from the government to 
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proceed with the drafting of amendments 
and that the draft legislation would be 
ready for tabling in Parliament in the fall 
of 1998. 


19.68 | Moreover, the Department has 
been developing a harmonized approach 
with the provinces and territories. It 
participates in the electronic commerce 
project of the Uniform Law Conference of 
Canada. Among other matters, the 
Conference planned to give final approval 
in late August to its draft Uniform 
Electronic Evidence Act, the substance of 
which was being incorporated into the 
proposed amendments to the Canada 
Evidence Act. 


Legal liability is a concern that requires 
further analysis and resolution 


19.69 Secure electronic commerce via 
the Internet provides an alternative means 
of conducting business. However, losses 
and damages can occur through ignorance, 
negligence, abuse or deliberate attempt, 
and can lead to legal liabilities. The 
delivery of government services 
electronically introduces new risks and 
exposure that can result in legal lability 
for the Crown. 


19.70 The need to address such 
potential liabilities has been generally 
acknowledged. In addition to legislation, 
the legal framework includes other 
instruments such as memoranda of 
understanding, contractual agreements and 
certification policies that form the basis 
for cross certification. Early analysis by 
the legal subcommittee showed that these 
instruments could be used to help limit the 
Crown’s exposure to the risk of liability. 
Nevertheless, the significance of legal 
exposure, if any, where no contractual 
relationship exists and third parties choose 
to rely on government information still 
needs to be determined. 


19.71 Consequently, the members of 
the government’s legal community have 
suggested that every application using the 
public key infrastructure be preceded with 
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a threat and risk assessment from a legal 
perspective. The assessment would 
identify and assess potential legal 
vulnerability so that it could be addressed 
accordingly, using various instruments in 
the legal framework. The Policy 
Management Authority committee also 
called for participating members of the 
GOC PKI project to put forward various 
scenarios for use in further analyzing the 
liability issues. 


19.72 The government should ensure 
that issues of potential liability are 
identified and addressed as it introduces 
new electronic commerce initiatives. 


The delivery of 
government services 
electronicaily 
introduces new risks 
and exposure that can 
result in legal liability 
for the Crown. 


Government’s response: New legislation 
will provide a framework for electronic 
commerce. The government is aware of 
concerns about liability and will ensure 
that departments are aware of any legal 
implications related to electronic 
commerce, including potential liability, 
and that they manage them accordingly. 


Action Needed to Develop 
Direction and Adequate Common 
Technology Infrastructures 


19.73 The Blueprint for Renewing 
Government Services Using Information 
Technology envisioned that the 
government would use information 
technology not only internally but to 
extend its services to the public 
electronically. Seamless access to services 
will require support from common 
technology infrastructures, so that the 
various technology platforms of different 
departments and agencies can 
interoperate, that is, operate readily with 
each other. 


Senior sponsor needed to lead electronic 
commerce in government 


19.74 To make electronic commerce the 
preferred way of doing business by 1998, 
it would be reasonable to expect that the 
government would follow its 1995 policy 
announcement with a strategic plan, led 
by a senior sponsor and co-ordinated with 
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As of early July 1998, 
no overall strategy or 
plan had been 
developed to update 
the 1995 strategy and 
to lead the government 
toward its goal. 


plans and actions in departments and 
agencies. 


19.75 Under the direction of the 
Treasury Board Secretariat, a strategic 
paper entitled “Electronic Commerce 
Commitment and Agenda for Action” was 
prepared in 1995 to lead the electronic 
commerce initiative. The paper set out a 
number of initiatives and identified a 
number of departments and working 
groups that would pursue them. Because 
of staff turnover and changes in priorities 
at the Secretariat, however, many 
initiatives were not continued beyond 
1996. 


19.76 As of early July 1998, no overall 
strategy or plan had been developed to 
update the 1995 strategy and to lead the 
government toward its goal. There was no 
statement of what is to be accomplished 
by 1998 or what being a model user by 
2000 entails. Furthermore, there was no 
common definition of electronic 
commerce in government. Departments 
and agencies have been developing 
various electronic initiatives, but no 
attempt has been made to co-ordinate 
them. Although an interdepartmental 
committee of deputy ministers requested 
that the progress of electronic commerce 
in government be monitored and specific 
information collected, there has been no 
attempt to do so. 


19.77. The primary effort at the 
Secretariat to advance electronic 
commerce in government has been by its 
PKI Task Force. Although cryptography 
services are critical to building confidence 
and trust in secure electronic commerce, 
they are not essential for all types of 
internal and external transactions of 
government. For example, among the four 
electronic initiatives that we examined, 
only one has a stated requirement for the 
services afforded by the government’s 
public key infrastructure. 


19.78 Departments and agencies are 
advancing their own electronic commerce 
initiatives. To advance electronic 


commerce government-wide, however, 
there is a need for a senior sponsor to 
assess the government’s progress and to 
set future direction. Without this sponsor, 
the government risks not achieving its 
policy objective and its commitment to 
become a model user. More significantly, 
the objective envisioned in its information 
technology Blueprint — to renew 
government services using information 
technology — may not be fully realized. 


Further efforts required for technology 
standards and guidelines 


19.79 The Chief Information Officer 
Branch of the Secretariat is responsible for 
setting government-wide standards. 
Standards set by the Secretariat are 
promulgated as Treasury Board 
Information Technology (TBIT) Standards 
and form part of Treasury Board policies 
for the government. 


19.80 Under the government’s public 
key infrastructure project, progress is 
being made in identifying and adopting 
open standards that will support cross 
certification and the use of cryptography 
services. For example, standards are being 
set for electronic directories in 
government so that certification 
authorities could include in them 
electronic certificates for access by 
government or public users. 


19.81 However, efforts to update TBIT 
Standards are slow. Technology standards 
are necessary to support common 
infrastructures so that different technology 
platforms can interoperate. Common 
infrastructures facilitate seamless access 
by the public to information and services 
from different departments and agencies. 
In Chapter 16 of our 1996 Report, we 
noted that many of the TBIT Standards 
were of questionable relevance and were 
being overtaken by de facto standards in 
the information technology industry. 
During the present audit, we noted that the 
need for standards to support common 
infrastructures in government had been 
discussed in many interdepartmental 
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forums. But action has yet to be taken to 
identify the types of standards that are 
necessary and the specific standards to be 
adopted for use by government. 


19.82 Through the work of an 
interdepartmental committee, the Treasury 
Board Secretariat has prepared an Internet 
Guide. The Guide is available on the 
Internet and provides technical guidance 
on creating websites. We found that the 
scope of the guidance could be broadened 
and that more emphasis could be placed 
on its use by departments and agencies. 


19.83. The Guide focusses primarily on 
technical advice for creating websites. It 
also contains some guidance to help users 
in departments and agencies address the 
government principle of “common look 
and feel” and “ease of use”. The principle 
of common look and feel is that websites 
should share certain common 
characteristics so that public users can 
readily identify that they are accessing 
government websites. The principle of 
ease of use is that websites should be easy 
to navigate, so that users can access 
information readily and efficiently. 
However, broader guidance on issues such 
as analyzing the impact of an Internet 
service on users and reviewing security 
and privacy concerns is not provided in 
this Guide or others. 


19.84 It also came to our attention that 
access to and use of the Secretariat’s 
Internet Guide were not tracked and there 
was no mechanism for feedback. Its length 
does not facilitate viewing on the Internet, 
and paper copies have not been 
disseminated. 


19.85 In 1995, Public Works and 
Government Services Canada received 
approval to establish a Canada Internet 
site that would provide a federal 
government presence on the Internet. It 
would also provide single-window access 
to government information and services. 
The concept of a single window was 
discussed in the government’s information 
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technology Blueprint in 1994 and was 
supported by the Information Highway 
Advisory Council in its 1995 report. The 
site, http://www.canada.gc.ca, became 
operational in December 1995. Its usage 
has increased approximately tenfold, from 
about 500,000 hits per month in early 
1996 to about five million hits per month 


in early 1998. Action has yet to be 


taken to identify the 
types of standards 
that are necessary and 
the specific standards 
to be adopted for use 
by government. 


19.86 The Canada Internet site met the 
initial requirement to provide a common 
point of access to government 
information. Yet it is not user-oriented and 
does not facilitate seamless access to 
various government services, especially 
those involving more than one department 
or agency. In most instances, a user would 
need to know specifically which 
department or agency provides which 
particular services in order to access them. 
Since its implementation, the site has 
remained static; changes have not gone 


- beyond routine maintenance. The 


Department advised us that it is aware of 
the site’s shortcomings and plans to 
address them. Subject to resource 
availability, it plans to adopt a 
client-driven orientation for the site. Since 
the audit, the Department has advised us 
that actions are under way to help users 
access information on the Canada site. 


19.87. Two and a half years after the 
policy statement on electronic commerce, 
there is no senior sponsor to set future 
direction and many issues remain to be 
addressed in order to achieve the common 
infrastructures needed. 


19.88 The government should appoint 
a senior sponsor for the goals of 
advancing electronic commerce in 
government and making government a 
model user of the information highway. 
In particular, it should consider: 


¢ defining electronic commerce in 
government; 


¢ developing strategy and setting 
direction for electronic commerce 
beyond 1998, identifying deliverables 
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and indicating when they may be 
completed; 


e assigning responsibilities, clearly 
defining the roles and terms of 
reference for those assigned and 
supporting them with appropriate 
authority and resources; 


e providing oversight and 
monitoring progress; and 


e reporting performance to 
Parliament on a periodic basis. 


19.89 The Treasury Board Secretariat 
should accelerate its efforts to identify 
and adopt technology standards that 
will support common infrastructures in 
government. It should ensure that 
standards to support the government’s 
public key infrastructure are in place 
and are followed when the crypto- 
graphy product becomes ready for 
deployment. 


Government’s response: In April of this 
year the government stated that to further 
enhance its commitment to the principles 
of service improvement, it has established 
within the Treasury Board Secretariat 
(TBS) a new sector with the mandate to 
focus on government-wide approaches to 
improving services to Canadians. The 
creation of the new sector was initiated as 
part of the Secretariat’s continued and 
concentrated effort to ensure that 
electronic commerce becomes the 
government's preferred way of doing 
business. The new sector is focussing on 
the government's priority of “connecting” 
Canadians and on making the information 
infrastructure available to all Canadians 
by the year 2000. The electronic service 
delivery channels will incorporate and 
make use of the public key infrastructure, 
the legal framework and the common 
technology infrastructures that are the 
focus of this audit. 


Recently the Prime Minister indicated the 
government's support for electronic 
commerce, saying that “the federal 


government is not alone in tackling issues 
related to electronic commerce....Together 
with business, consumers and other 
governments, we are making progress on 
various electronic commerce issues such 
as taxation, intellectual property, privacy, 
legal frameworks and cryptography 
policy.” 


It should be noted that sponsorship is a 
corporate responsibility of the 
government, with departments playing a 
key role. Senior sponsorship is the 
responsibility of more than just one 
department or deputy. Electronic 
commerce is a new way of doing business 
and demands new approaches. The issues 
are horizontal and touch all departments. 
It is important to recognize this horizontal 
responsibility. 


The government's progress in adopting 
electronic commerce will be reported in 
the Treasury Board Secretariat's annual 
Report on Plans and Priorities and Report 
on Performance. 


The government is addressing the need for 
information technology standards. Key 
technology standards for secure 
messaging, directories, “smart cards” and 
documents are being examined and 
adopted or proposed for adoption. The 
establishment of additional standards to 
support PKI will be undertaken as the 
need is identified, based on their relative 
priorities. An internal review of the 
standards needed for implementation of 
PKI, electronic commerce and service 
delivery will be undertaken in the fall. 
This review is expected to identify a 
number of standards projects that should 
be initiated or reactivated. 


Electronic Initiatives in 
Government — A Profile 


19.90 As part of the audit, we selected 
and reviewed four electronic commerce 
initiatives in the government. The purpose 
of the review was to identify good 
practices and lessons learned from the 
government’s early experience in 
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delivering services via the Internet. A 
brief description of the initiatives and a 
summary of our findings appear in 
Exhibit 19.4. 


19.91 All four initiatives use the 
Internet in delivering government services 
to individuals or Canadian businesses. 
Three of them charge fees for services 
rendered. When fully implemented, two 
initiatives will be used to complement 
existing hard copy services; the other 
initiatives will become the only means of 
access to those services. 


19.92 Some initiatives are in pilot 
mode, while others have been fully 
implemented. The initiatives range from 
the simple to the complex. They require 
security features in varying degrees, from 
being able to authenticate the identity of 
users to protecting the integrity of 
transmitted data and collecting user fees. 


Good practices and potential lessons 
learned 


19.93 One initiative involved the 
re-engineering of a business process in 
co-operation with stakeholders. The 
National Energy Board is developing the 
Electronic Regulatory Filing system with 
a group of private sector companies it 
regulates and the Ontario Energy Board. 
When implemented, it will change the 
process of filing submissions for Board 
hearings to one that receives submissions 
electronically. It will also allow potential 
interveners 1n hearings to download 
relevant portions of the documents for 
their use. Currently, a single hearing often 
necessitates the copying and distribution 
of hundreds of thousands of pages of the 
document by the company filing the 
submission. In developing the initiative, 
the Board took into consideration the 
impact on program participants (regulated 
oil and gas companies), other levels of 
government (provincial regulators) and 
other interveners (the public, or other 
private sector firms that could be affected 
by a submission). 


Electronic Commerce: Conducting Government Business 


19.94 Two of the initiatives we looked 
at considered the opportunity for private 
sector partnership and are now involving 
private sector companies in delivering 
services electronically. Through its MERX 
system, Public Works and Government 
Services Canada outsources the provision 
of information to suppliers for purposes of 
tendering bids for procurement contracts. 
The Department and participating 
provinces provide the content; the private 
sector firm develops and maintains the 
website and the billing system to charge 
and collect fees from users. 


19.95 In collecting user fees for its 
Online statistics application, Statistics 
Canada uses the service of a financial 
institution to handle all credit card 
transactions. A solution internal to 
government would have required the use 
of public key cryptography services. It 


. would also have required Statistics 


Canada to certify the users directly or to 
cross certify with the certification 
authorities for those users. Further, as we 
have noted, approval to deploy the 
technical product for the government’s 
public key infrastructure is not expected 
until December 1999 and this would have 
delayed the implementation of Statistics 
Canada’s initiative. As of July 1998, the 
Online statistics initiative had been 
functional for about two years. 


19.96 The government’s Enhanced 
Management Framework for managing 
information technology projects calls for 
the preparation of business cases. 
Electronic initiatives can be developed for 
various purposes, such as to experiment 
with new service delivery mechanisms, to 
streamline operations and reduce costs, or 
to respond to business pressures and user 
demands. Without a business case that 
states the desired benefits of an initiative, 
it would be difficult to later assess its 
merits. In the absence of cost estimates at 
the start, cost overruns and broadening of 
scope would be difficult to control. With 
one exception, we found that business 
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cases had not been prepared for the 
initiatives we reviewed. 


19.97 We also found that none of four 
departments and agencies has conducted 
threat and risk assessments of the 
initiatives. Threat and risk assessments are 
usually conducted to determine the level 
of security required for transactions and to 
assess alternative solutions for security. 
They help to identify the most appropriate 
and cost-effective security solution. 


19.98 In our view, existing electronic 
commerce initiatives can provide a wealth 
of information for other projects. In 
addition to learning from the experience 
gained in these initiatives, other projects 
may use specific features and elements of 
them. For example, other government 
organizations may also have business 
applications that require secure filing of 
documents, or may also charge user fees 
for services. Although there is a list of 
electronic commerce initiatives in 
government, no group has been given the 
responsibility to keep it up-to-date, or to 
review the initiatives and disseminate the 
experience gained to departments and 
agencies. 


19.99 The Treasury Board Secretariat 
should ensure that an inventory of 
electronic commerce initiatives in 
government is maintained and updated 
periodically. It should analyze the 
initiatives for good practices and lessons 
learned, and share the information with 
departments and agencies so that they 
can take advantage of others’ 
experience. 


Government’s response: We agree with 
the need to ensure that information on 
electronic commerce initiatives is shared 
across the government in order to take 
advantage of the experience gained by 
departments. This recommendation must 
be balanced with the need to not 
overburden departments with requests for 
more reporting. Pathfinder projects are 
currently highlighted by TBS. An informal 
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inventory will continue to track initiatives 
and their progress. 


Conclusion 


19.100 The government is making 
progress in the areas we examined and is 
moving toward conducting business via 
the Internet by addressing barriers to 
electronic commerce. 


19.101 In particular, it is developing a 
technical solution to concerns about the 
security of transactions conducted on open 
public networks. An interim product is 
available and is expected to receive 
government approval by the end of 1998. 
The management framework for this 
cryptography project is being developed. 


In our view, existing 
electronic commerce 
initiatives can provide 
a wealth of information 
for other projects. 


19.102 In considering the legal 
framework, the government has identified 
language containing a paper bias in all 


resolution. Through an interdepartmental 
committee, the Treasury Board Secretariat 
has produced an Internet Guide that offers 
technical advice to departments and 
agencies developing websites, and there is 
a Canada site that provides a federal 
government presence on the Internet. 


19.103 At the same time, we identified 
several key risks that could undermine the 
public key infrastructure project that the 
government has undertaken to support its 
secure electronic commerce. We found 
that the business process using the Internet 
and the development of related computer 
applications are lagging behind the 
progress of the public key infrastructure 
project, creating a risk that it may be 
underutilized when it is completed. The 
need to certify public users and the 
implications of doing so present another 
risk. If not addressed on a timely basis, it 
could severely limit the use of the 
infrastructure to support external service 
delivery. Furthermore, we noted some 
challenges to the project’s development 
and implementation that need to be 
overcome. 
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19.104 We also noted that electronic common infrastructures are in place to 
commerce can have important support seamless delivery of services 
implications for the government’s legal across departments and agencies. Strategy 
liability; further analysis and resolution of and direction are needed to define the 
this issue are required. government’s objectives and goals for 
electronic commerce beyond 1998. 
19.105 Moreover, we observed that a Further action will also be required to 
senior sponsor is needed to advance meet the government’s commitment to 
electronic commerce in government. become a model user of the information 


Many issues remain to be resolved before highway by 2000. 
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Objective and Scope | 


The audit focussed on the conduct of government business via the Internet, a predominant area of growth in 
electronic commerce. Government business includes administration, operations, and delivery of programs and 
services. The use of the Internet in conducting government business is an important part of the government’s 
policy of making electronic commerce its preferred way of doing business. The government’s commitment to 
serve as a model user of the information highway supports the Information Highway Advisory Council’s 
recommendation and forms part of the platform toward Canada’s goal of becoming a world leader in | 
electronic commerce. 


The audit objective was to assess the government’s progress in three key areas: 


e the development and implementation of a public key infrastructure for the federal government as a 
measure for secure electronic commerce; 


e the review of and changes to the legal framework in support of transacting business electronically; and 


e the implementation of common infrastructures to support government administration, operations and 
delivery of services via the Internet. 


The audit sought to identify and highlight issues in these areas that require resolution to help advance the use 
of the Internet in government operations, both internally and externally. We did not audit other emerging 
policies such as those relating to privacy, cryptography, intellectual property and taxation. Nor have we 
examined the government’s role and efforts in broadening public access and enhancing skills and awareness 
of the information highway. 


We also examined four electronic commerce initiatives being pilot-tested or operating with the use of the 
Internet in government service delivery. They include trademarks registration at Industry Canada, electronic 
regulatory filing at National Energy Board, the MERX tendering service at Public Works and Government 
Services Canada, and Online statistics (CANSIM and trade data) at Statistics Canada. 


Criteria 
The general criteria used in the audit were as follows: 
Security 


e Clear roles and responsibilities should be established for ensuring proper safeguards for electronic 
commerce within and outside government. 


e Risks and issues concerning electronic commerce, specifically confidentiality of information and 
authentication of senders/recipients, should be identified and addressed in a cost-effective manner. 


Legal framework 


e The legal framework should provide for appropriate definition and interpretation of records and 
signatures in support of the use of electronic commerce in government. 
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Common infrastructures 


e Government-wide standards and guidelines should be in place to ensure support of electronic commerce 
with common technological infrastructures that offer reliability, interoperability and scalability. Policies 
and guidelines to effect a government-wide approach to providing electronic services should exist and be 
readily accessible by the intended community and easy to use. 


e Measures should be in place to ensure proper conformance to standards and policies. 
Selected electronic commerce initiatives 


e Electronic commerce initiatives should be developed and managed to meet business needs with due 
regard to security, legal requirements and interoperability. 


Audit Team 


Assistant Auditor General: Doug Timmins 
Principal: Nancy Cheng 
Directors: Guy Dumas and Tony Brigandi 


Joe Lajeunesse 


For information, please contact Nancy Cheng. 
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Chapter 20 


Preparedness for Year 2000 


Government-Wide Mission-Critical 
systems 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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Preparedness for Year 2000 


Government-Wide Mission-Critical 
systems 


Main Points 


20.1 The Year 2000 problem was caused by the past programming practice of coding the year with two digits 
rather than four. On 1 January 2000, certain computer applications and devices could fail or malfunction. In 1997, 
we reported that the government’s rate of progress in preparing for Year 2000 had generally been slow; if it were 
to continue at the same rate, government systems probably would not be ready in time. 


20.2 In 1998, we audited some key computer systems and devices that support six mission-critical functions in 
government. The functions were inspection services to contribute to food safety; emergency assistance and 
support; income security services; First Nations transfer payments and trust funds; border crossing services for 
commercial goods; and law enforcement services. We concluded that as of 30 June 1998, several key systems 
supporting three of those functions remained at risk of not becoming fully compliant before 2000. 


20.3 We observed that the government has accelerated the pace of its efforts since our 1997 audit and has 
made progress in various areas, including: 


e ranking Year 2000 as a top priority; 


¢ monitoring government-wide progress of systems and devices that support mission-critical functions 
in government; 


e advancing certain common horizontal initiatives; and 
e engaging the concern of ministers and advising them of the urgency of Year 2000 efforts. 


20.4 However, our review of the 30 June 1998 survey results summarized by the Treasury Board Secretariat 
showed “good” progress in only four departments, accounting for 7 out of 48 government-wide mission-critical 
functions. Nine departments and agencies responsible for 18 mission-critical functions had a score of 50 percent or 
less for completion of the work needed to make the systems supporting the functions compliant. 


20.5 We concluded that as of June 1998, various government systems supporting mission-critical functions 
remained at risk. With 18 months left, we are very concerned that some essential government services may be 
interrupted at the start of 2000. 


20.6 To ensure that mission-critical functions will continue, we recommend that the focus on Year 2000 be 
sustained, that the Treasury Board Secretariat consider strategic intervention as appropriate, and that contingency 
plans be prepared and tested for those systems that remain at risk. We also recommend that further action be taken 
to advance common horizontal initiatives and that reporting of Year 2000 information to Parliament be improved. 
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Introduction 


20.7 The Year 2000 problem, also 
known as the millennium bug, Y2K and 
century date change, refers to the potential 
for computer systems error, malfunction 
or failure caused by the past practice of 
representing the year with a two-digit 
code. Due to the extensive use of the date 
code, the information technology industry 
has estimated that some 90 percent of 
systems and codes will be affected by 
Year 2000. 


20.8 Given the ever-increasing 
reliance on information technology, Year 
2000 represents a business risk, not just a 
technical problem. It is a threat to all 
organizations, including businesses, 
non-profit organizations, schools, 
hospitals and all levels of government. 


20.9 In 1997, we audited the federal 
government’s overall state of preparedness 
for Year 2000. We concluded that as of 30 
April 1997, the government’s rate of 
progress in addressing the Year 2000 
threat had generally been slow. 


20.10 We were concerned that if 
progress were to continue at the same rate, 
it would likely be too slow to ensure that 
government systems would be ready in 
time. We concluded that Year 2000 
remained a serious threat to essential 
programs and services and that urgent and 
aggressive action was required. 


20.11 We recommended that the 
Treasury Board Secretariat give top 
priority to monitoring progress. In 
particular, we recommended that it 
identify the systems considered to be 
mission-critical to the government as a 
whole and intervene strategically as 
appropriate, to ensure that they would be 
ready before 2000. We also emphasized 
the need for contingency plans. Other 
recommendations included: 


e ranking Year 2000 as a top priority; 
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e affirming the sustained commitment 
and support of senior management; 


¢ informing Parliament of the potential 
impact of Year 2000 and reporting on 
progress toward making systems 
compliant; and 


e accelerating common initiatives that 
support Year 2000 work in departments 
and agencies. 


Focus of the audit 


20.12 The current audit focussed on the 
government’s progress in making its 
mission-critical systems compliant for 
Year 2000. 


20.13 Weused the results of the 
Treasury Board Secretariat’s 30 June 1998 
survey to obtain an update of the 
government’s progress in making 
compliant the systems that support its 
mission-critical functions. We examined 
several key systems and applications 
that support six government-wide 
mission-critical functions. We also 
followed up on action taken by the 
government to address our 1997 
recommendations. 


20.14 Further information about the 
audit objective, scope and criteria can be 
found at the end of the chapter in About 
the Audit. 


Observations and 
Recommendations 


Examination of Systems That 
Support Government-Wide 
Mission-Critical Functions 


20.15 In July 1997, the Treasury Board 
Secretariat prepared an initial list of 
government-wide mission-critical 
functions — those considered essential to 
delivering programs and providing 
services. The list was circulated to all 
deputy ministers and heads of agencies for 
feedback. The departments and agencies 


The current audit 
focussed on the 
government's 


progress in making its 


mission-critical 


systems compliant for 


Year 2000. 
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As of June 1998, 48 
government-wide 
mission-critical 
functions had been 
identified. 


responsible for the functions were also 
requested to provide a brief impact 
statement on each of the functions. 


20.16 In the fall of 1997, the list 
included 19 departments and agencies that 
provide 44 mission-critical functions of 
the government. Those functions became 
the focus of the Secretariat’s subsequent 
survey efforts. Year 2000 work on them 
receives priority access to resources such 
as funding, and to procurement 
arrangements. As of June 1998, 48 
government-wide mission-critical 
functions had been identified; they are 
provided by 21 departments and agencies 
and two Crown corporations (Exhibit 
20.1). 


20.17 In assessing the government’s 
progress, we selected for examination six 
government-wide mission-critical 
functions by applying the criteria used in 
our 1997 audit. These functions include: 


e inspection services to contribute to 
food safety; 


e emergency aid to civil powers and 
assistance and support in the event of 
natural disaster or loss of civilian 
infrastructure; 


e income security services to seniors; 


e transfer payments to First Nations 
and management of their trust funds; 


e border crossing services for 
commercial goods, including collection of 
and accounting for customs duties and the 
goods and services tax; and 


e law enforcement services for the 
safety and security of the public. 


20.18 | Government-wide 
mission-critical functions are supported by 
many computer programs, systems and 
devices that vary significantly in size and 
complexity. In early 1998, the Chief 
Information Officer at Treasury Board 
Secretariat estimated that over 1,000 
systems support these functions. In each 


of the six functions we selected, we 
attempted to identify for audit the most 
critical systems, including embedded 
devices (those that require micro- 
processors to make them operational). 


20.19 In some functions we audited the 
lead system; in others, we selected and 
audited up to three key applications and 
systems. All the selected systems and 
devices are considered necessary to 
support the mission-critical functions but 
not always sufficient to ensure that the 
functions will continue. Our findings 
pertain only to the systems and devices 
that we audited and are not representative 
of overall progress in the organizations 
where they operate. Our findings reflect 
the status of progress at 30 June 1998 

(31 August 1998 for emergency assistance 
and support). 


Canadian Food Inspection Agency — 
inspection services for food safety and 
animal and plant health 


20.20 Food inspection services help 
maintain Canada’s high standards of 
quality and safety for food, agricultural 
inputs, and animal and plant health. The 
Canadian Food Inspection Agency 
depends extensively on information 
technology to carry out inspection, testing 
and certification of food products. Its 
systems that support food inspection are 
mission-critical because their potential 
failure or malfunction could result in 
health and safety concerns. 


20.21 The Canadian Food Inspection 
Agency was created in 1997 to 
consolidate the food inspection and 
animal and plant health activities of three 
federal departments. To address the Year 
2000 threat, the Agency decided to 
develop new information systems, 
including the Multi-Commodity Audit 
Program (MCAP) and Laboratory Sample 
Tracking Systems (LSTS), in the place of 
existing systems from three departments 
(Exhibit 20.2). Further, the Agency 
needed to test and replace laboratory 
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Department, Agency 
} or Crown Corporation 


Function 


Atomic Energy of Canada Limited 


control, safety, monitoring and facilities management 


Canadian Broadcasting Corporation 


communications and broadcasting system 


Canadian Food Inspection Agency 


food production and inspection 


Canadian Heritage (Parks Canada) 


maintenance management system 


Canadian Security Intelligence 
Services 


security intelligence 
security screening 


Citizenship and Immigration Canada 


managing access to Canada 


Communications Security 
Establishment 


foreign intelligence and information technology 
security 


Correctional Service Canada 


offender reintegration 


_ Environment Canada 


environmental forecasting system 


Fisheries and Oceans 


Canadian hydrographic service 

icebreaking operations 

marine communications and traffic services 
fleet management 

marine navigation services 

rescue, safety and environmental response 


Foreign Affairs and International 
Trade 


consular affairs 

export and import controls 
Canadian passport office 
network (messaging system) 


Health Canada 


laboratory centre for disease control 
therapeutic products program 

food program 

environmental health program 
medical services 


Human Resources Development 
Canada 


income security 
employment insurance 


Indian and Northern Affairs Canada 


band support funding 


Department of Justice 


family orders and agreements enforcement 


National Defence 


defending Canada 

defending North America 

contributing to international safety 
support to operations 

support to broad government programs 


Natural Resources Canada 


Public Works and Government 
Services Canada 


geomagnetic monitoring 


aeronautical and technical services 
seismic monitoring 


public service compensation 
Receiver General services 
processing government financial transactions 


Exhibit 20.1 


Government-Wide 
Mission-Critical Functions 


(continued) 
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Exhibit 20.1 


(continued) 


Source: Treasury Board 
Secretariat, June 1998 


Department, Agency 
or Crown Corporation Function 
Royal Canadian Mounted Police e law enforcement 
Revenue Canada e social income redistribution 
e income tax processing 
e customs border services and trade administration 
Tax Court of Canada e appeals management system 
Transport Canada e transport regulation 
Veteran Affairs Canada e health care 
e pensions and allowances 


equipment where necessary to make it 
Year 2000 compliant. 


20.22 We noted that implementation 
dates for the MCAP systems were delayed 
by about four months from earlier plans. 
The Year 2000 effort for laboratory 
equipment also experienced delays. 


20.23 We found that the Agency had 
identified interdependencies among its 
Year 2000 efforts on individual systems. 
However, analyses of these 
interdependencies for completion timeline 
and critical path modelling were 
incomplete. We were advised by the 
Agency that it had attempted to conduct 
these analyses but had not been able to 
complete the task, due to a number of 
factors such as time and costs. As a result, 
there is no critical path for the Agency’s 
overall Year 2000 project. Further, as of 
June 1998, it had yet to begin reviewing 
its interdependencies with other countries. 


20.24 Although the Agency has a 
defined protocol for testing and 
certification of its information systems 
and technology infrastructure and has 
developed an overall test strategy, it has 
not prepared test plans for the individual 
systems. As of 30 June 1998, it had yet to 
finish establishing a Year 2000 test 
environment. These environments are 
established to simulate processing in 
2000. Systems are not considered fully 
compliant for Year 2000 purposes until 


they have been tested successfully in this 
type of environment. 


20.25 Some application systems can 
have failure dates prior to 2000. We 
observed that the Agency’s analysis of the 
possibility of early failure of its 
application systems was insufficient and 
required further work. We noted that one 
application in LSTS that experienced 
failure due to the date code had not been 
identified in the Agency’s analysis. 


20.26 The Canadian Food Inspection 
Agency has over 2,000 pieces of 
laboratory equipment. Many of them 
contain microprocessors that can be 
difficult to detect and test, complicating 
the work of making those systems 
compliant for Year 2000. In most cases, 
communication with and guidance from 
the suppliers will be required. The Agency 
contacted suppliers of its laboratory 
equipment but, when we completed our 
audit, it had not received responses from 
about two thirds of them. 


20.27 The target completion date for 
the Agency’s Multi-Commodity Audit 
Program (MCAP) and Laboratory Sample 
Tracking Systems (LSTS) is June 1999. 
While this meets the milestone date 
stipulated by the Treasury Board 
Secretariat, it does not provide room for 
further slippage or for unexpected 
problems with various data interfaces 
among systems. The testing and 
replacement of laboratory equipment were 
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planned for completion at 30 October 
1998. Given that the needed vendor 
responses on most of the Agency’s 
laboratory equipment had yet to be 
received by the end of the audit, 
completion by October 1998 was unlikely 
and no new target date had been set. Thus, 
in our view, the MCAP and LSTS systems 
and the laboratory equipment remained at 
risk of not being fully compliant in time 
for Year 2000. 


20.28 The Year 2000 project office at 
Treasury Board Secretariat requested 
departments and agencies with systems 
that support government-wide 
mission-critical functions to complete and 
submit a Year 2000 risk assessment by 

31 August 1998, to determine if 
contingency plans would be necessary. At 
the time of our audit, the Canadian Food 
Inspection Agency was planning to assess 
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its Year 2000 risks and prepare detailed 
contingency plans by the end of August 
1998. Subsequent to the audit, the Agency 
advised us that it expected the two 
systems and its laboratory equipment to be 
fully compliant in time for Year 2000. 


National Defence — emergency 
assistance and support 


20.29 National Defence is responsible 
for five government-wide mission-critical 
functions. In early 1998, as part of its 
Operational Readiness program, it 
commissioned a series of impact analyses 
for 12 representative missions that it needs 
to undertake from time to time. One such 
national mission is to provide emergency 
aid to civil powers and assistance and 
support in the event of natural disaster or 
loss of infrastructure. The 1997 Red River 
flood in western Canada and the 1998 
eastern Canada ice storm are examples of 


Systems selected for audit e Multi-Commodity Audit Program (MCAP) — audit 
inspection module 


e Laboratory Sample Tracking Systems (LSTS) 


e laboratory equipment — level 4 laboratory, disease control 
laboratory in Winnipeg 


| Relationship to 
mission-critical function 
that they support 


| 
| 


MCAP supports the audit and inspection function, replacing 
mission-critical systems to support inspection of all commodities 
including meat and fish. The LSTS supports sample tracking 
and manages analyzed results. The laboratory equipment 
supports the analysis of food and other samples. 


At the time of our 
audit, the Canadian 
Food Inspection 
Agency was planning 
to assess its Year 2000 
risks and prepare 
detailed contingency 
plans by the end of 
August 1998. 


Exhibit 20.2 


Canadian Food Inspection 
Agency — Inspection Services 
for Food Safety and Animal 
and Plant Health 


Two mission-critical subsystems of MCAP have about 100,000 
lines of code each. Three mission-critical subsystems of LSTS 
have between 100,000 to 200,000 lines of code each. The 
systems operate in a client/server environment. The Agency has 
over 2,000 pieces of laboratory equipment that can be 
susceptible to the Year 2000 problem. 


Systems description 


Status (30 June 1998) For subsystems of audit and inspection module in MCAP, some 
were in development phase; some were being tested. All | 
subsystems in LSTS were in development phase. For its 
laboratory equipment, the Agency was in the process of 
following up with equipment manufacturers and vendors and 


testing embedded devices. 


Testing plans and strategy The Agency has a Testing and Certification Protocol for its 
information systems, information technology infrastructure and 
embedded systems. It has an overall test strategy but individual 


test plans were not available. 


June 1999 for MCAP and LSTS; October 1998 for laboratory 
equipment 


Planned completion dates 
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the events that this type of mission would 20.31 During the planning phase, the 
support. Department found that the Joint 
Establishment and Movement 
Management System (JEMMS) had been 
developed in 1996 using programming 
tools that were Year 2000 compliant. The 
JEMMS operates on personal computers. 
Although the personal computers at 
National Defence Headquarters are Year 
2000 compliant, in August 1998 those at 
the Department’s bases and stations were 
being tested and replaced as necessary. 
Also in August 1998, the systems manager 
and the director certified that JEMMS 
does not use dates. Since the audit, the 
Department has advised us that JEMMS 
will be considered compliant after the 
hardware at its bases and stations are 
confirmed or made compliant. 


20.30 The impact analysis for this 
representative mission was completed in 
late June 1998. The analysis identified 31 
essential systems, 17 of which were 
categorized as “very high priority”. We 
selected three of the 17 systems for 
examination — Joint Establishment and 
Movement Management System 
(JEMMS), Automated Defence Data 
Network (ADDN) and National Materiel 
Distribution System (NMDS). The 
JEMMS generates task force movement 
tables to support the movement of staff 
and equipment in a planned mission. The 
ADDN provides messaging capability 
seven days a week and 24 hours a day to 


facilitate secure and reliable 20.32 By August 1998, programming 
communication in times of peace and war. _ codes in the Automated Defence Data 
The NMDS produces documents for Network (ADDN) had been verified and 
distribution of materiel, and tracks were undergoing testing. National 
shipments (see Exhibit 20.3). Defence has requested and received 
Exhibit 20.3 Systems selected for audit e Joint Establishment and Movement Management System 
(JEMMS) 


National Defence — Emergency 


Assistance and Support e Automated Defence Data Network (ADDN) 


e National Materiel Distribution System (NMDS) 


Relationship to The three systems support the movement of personnel and 
mission-critical function materials needed for planned missions and facilitate 

that they support communication by providing a secure messaging capability. 
Systems description The JEMMS system is a stand-alone PC-based application that 


was developed in 1996. The ADDN has two nodes in Ottawa. 
It communicates with 80 widely distributed personal computers 
and has data links to other agencies and allied countries. The 
NMDS operates on 45 minicomputers that are connected via the 
Department’s integrated data network. It supports about 250 
terminals that are located at two supply depots and all 
departmental shipping and receiving areas. The NMDS system 
is also located abroad and has mobile systems that are 
connected via satellite. 


Status (31 August 1998) The JEMMS was certified as Year 2000 compliant in August 
1998. Code check for ADDN was completed and testing was 
continuing; non-compliant ADDN personal computers were 
identified. The NMDS system was in conversion. 


Testing plans and strategy Test plans completed for ADDN. Although testing checklist 
exists, test plans have not yet been prepared for NMDS. 


Planned completion dates 28 February 1999 for ADDN; 31 March 1999 for NMDS 
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confirmation of Year 2000 compliance 
from suppliers of the core software and 
hardware. About 15 older personal 
computers have been identified as 
non-compliant. The Department plans to 
apply a software solution to address the 
problem and to not replace the computers 
at this time. At the end of our audit, it was 
expecting to complete the software 
correction by November 1998. According 
to departmental officials, the ADDN 
system and other messaging systems in the 
Department are expected to be phased out 
and replaced by a new departmental 
message-handling system in 2001. 


20.33 The technology platform for the 
National Materiel Distribution System 
(NMDS), including hardware, software 
and the data base, has been assessed as 
non-compliant and needs to be replaced. 
In addition, the existing hardware that 
supports NMDS has experienced more 
frequent breakdowns in recent years. The 
Department plans in April 1999 to phase 
out its integrated data network, the 
network used by NMDS. 


20.34 The Department prepared 
detailed project plans to redevelop NMDS 
and reinstall it on a compliant technology 
platform; at the end of our audit, the 
redevelopment work was on schedule. 
Test plans for NMDS are not scheduled to 
be completed before February 1999. 


20.35 Some risks of delay are involved 
in replacing non-compliant data terminals 
and procuring and installing work stations 
to access the Department’s wide area 
network. This could delay in turn the 
completion of Year 2000 conversion and 
testing of NMDS. 


20.36 In our view, the risks facing the 
three systems that we examined are not 
significant in comparison with the risks to 
other systems in the Department or to 
others we examined in other departments 
and agencies. Year 2000 risks are higher 
in the Department’s large information 
technology projects and the large number 
of embedded devices. Over 80 percent of 
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its systems are embedded systems; when 
we completed our audit, its inventory list 
was still being updated. 


20.37 Under the Department’s 
Operational Readiness program, impact 
and risk analyses for representative 
missions were to continue in the fall of 
1998. As of 31 August 1998, the 
Department considered that no 
contingency plans were necessary for the 
three systems we audited. Subsequent to 
the audit, the Department indicated that it 
was planning a number of activities to 
reduce risk in its embedded systems. It 
also advised us that it would prepare 
contingency plans for the three systems 
and develop business continuity plans 
under the Operational Readiness program. 


Human Resources Development Canada 
— income security services 


20.38 Income security services at 


‘ Human Resources Development Canada 


include the Canada Pension Plan (CPP) 
and the Old Age Security program. For 
many seniors who depend on benefit 
payments as part of their retirement 
income, services that provide those 
payments are essential. The CPP provides 
3.7 million benefit payments to eligible 
contributors and to surviving spouses and 
dependent children (Exhibit 20.4). 


20.39 Over the past few years, under an 
initiative to redesign the information 
systems for its income security programs, 
Human Resources Development Canada 
compiled a full inventory of the CPP 
system. While the redesign project was 
under development, the Department 
started Year 2000 repair work on the 
existing CPP system as a contingency 
measure. When the redesign project was 
halted in late 1997, the Year 2000 strategy 
focussed on repairing the CPP system. 
The conversion strategy involved using a 
combination of a four-digit year code and 
program logic for two-digit codes. The 
repair work on the three subsystems was 
concurrent, and completion was planned 
in sequence — Earnings, Benefits and 
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Exhibit 20.4 


Human Resources 
Development Canada — 
Income Security Services 


then Payments. The Department also used 
a phased approach to repairing and testing 
the subsystems. 


20.40 As of June 1998, repair work on 
the Earnings subsystem was almost 
complete and the repaired codes were 
undergoing tests for quality assurance and 
acceptance. We noted a minor slippage in 
the repair work for the Benefits 
subsystem: 73 percent of the programming 
codes had been converted by the end of 
June 1998. Code conversion for the 
Payments subsystem was about one month 
behind. At 30 June 1998, just under half 
the codes had been converted and the 
completion date had been rescheduled to 
mid-November 1998. 


20.41 The Department completed its 
analysis phase of the three CPP 
subsystems in February 1998. According 
to its plans prepared in late February, 
quality assurance and acceptance testing 
of the Benefits and Payments subsystems 
was to occur Over approximately 

14 months. The planned 14 months 
included testing for CPP legislative 
changes passed by Parliament in 
December 1997. As work on those two 
subsystems progressed, the plans were 
revised each month. As of June 1998, 
about 20 percent of the Benefits 
subsystem had been tested for quality 


assurance and acceptance and those tests 
had not begun for the Payments 
subsystem. 


20.42 The testing of systems in a Year 
2000 test environment is a critical step in 
ensuring compliance and readiness. Thus, 
once established, such a test environment 
becomes an important resource and often 
its use is scheduled to full capacity. 
Human Resources Development Canada 
has arranged to test its three CPP 
subsystems in a Year 2000 test 
environment (called MVSK), which is 
managed by another department. The 
testing of the CPP system in that 
environment is scheduled from 
mid-October to 31 December 1998. In 
addition to supporting its own Year 2000 
testing, the department managing MVSK 
has set up a tight schedule to support its 
other client departments and agencies. 


20.43 Before testing in the Year 2000 
environment, code conversion and quality 
assurance and acceptance testing need to 
be completed. There is a risk that those 
steps may not be finished in time for the 
MVSK testing. In the event that the CPP 
subsystems are not fully ready, Human 
Resources Development Canada could 
miss its scheduled MVSK testing and have 
difficulty rescheduling. As of June 1998, it 
appeared that the Department would need 
to compress its quality assurance and 


Systems selected for audit 


Canada Pension Plan (CPP) system — Earnings, Benefits and 
Payments subsystems 


Relationship to 
mission-critical function 
that they support 


The system manages information relating to the Canada Pension 
Plan and provides benefit payments to eligible contributors or 
their beneficiaries. 


Systems description 


environment. 


The three subsystems are legacy systems with about 2.4 million 
lines of code. They operate in a mainframe COBOL 


Status (30 June 1998) 


in conversion. 


The Department was testing the Earnings subsystem and a 
portion of the Benefits subsystem for quality assurance and 
acceptance. Portions of Benefits and Payments subsystems were 


Testing plans and strategy 


Test strategy exists; test plans for Year 2000/MVSK 
environment have yet to be completed. 


Planned completion dates 


31 December 1998 
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acceptance testing process to meet the 
MVSK schedule. Compressed testing 
could increase the risk of failure in the 
Year 2000 test environment; it could also 
cause functionality problems after the 
converted subsystems are implemented. 


20.44 Quality assurance and acceptance 
testing is an essential step. We noted that 
as of June 1998 the testing team was not 
fully staffed and the quality assurance and 
acceptance testing of some functions in 
the CPP subsystems had yet to be 
assigned. Although detailed plans for 
quality assurance and acceptance testing 
were in place, plans for testing in the Year 
2000 environment (MVSK) were not 
complete. 


20.45 System interfaces for the CPP 
system were identified during the redesign 
project and were subsequently confirmed 
as part of the Department’s Year 2000 
project. Work on the interface of the 
Earnings subsystem with data on CPP 
contributions has been co-ordinated with 
another department. Human Resources 
Development Canada has requested and 
received a copy of the Year 2000 plans for 
the government's payment system, which 
interfaces with its Payments subsystem; 
periodic meetings have been held since 
December 1997 to co-ordinate testing 
activities between the two systems. 


20.46 Human Resources Development 
Canada’s plan is to make the CPP system 
fully compliant by December 1998, 
leaving up to one year to address any 
slippage or problems encountered during 
testing. As of 30 June 1998, quality 
assurance and acceptance testing was 
ongoing; it was uncertain whether testing 
in MVSK would be completed 
successfully by 31 December 1998. In our 
view, there remains a risk that some of the 
CPP subsystems may not be fully 
compliant before 2000. If the CPP 
subsystems fail to pass MVSK testing in 
1998, the Department has indicated that it 
will retest the changes and corrections for 
quality assurance and acceptance and then 
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retest in MVSK. In that event, it will 
implement a detailed contingency plan to 
ensure that the essential service of 
providing CPP benefit payments will 
continue in 2000. 


20.47 As we completed our audit, 
Human Resources Development Canada 
was intending to prepare a risk assessment 
by the end of August 1998, as requested 
by the Treasury Board Secretariat. The 
Department indicated that it was confident 
that the CPP system would be compliant 
by the end of December 1998. It also 
indicated that, in any event, it would 


finalize detailed contingency plans. Human Resources 


Development Canada 
indicated that it would 
finalize detailed 
contingency plans. 


Indian and Northern Affairs Canada — 
First Nations transfer payments and 
trust funds 


20.48 First Nations transfer payments 
support basic services on reserves, such as 
education, social assistance, housing and 


‘other capital facilities. Transfer payments 


are also made pursuant to comprehensive 
land claims and specific claims. In 
fulfilling the government’s fiduciary 
responsibilities, Indian and Northern 
Affairs Canada also provides trust 
services, including the management of 
Indian moneys as stipulated in the Indian 
Act. These are essential services for 
Aboriginal peoples and a mission-critical 
function of the government. 


20.49 The Transfer Payment 
Management System (TPMS) and the 
Trust Fund Management System (TFMS) 
are the primary systems supporting this 
function. We also examined the 
Departmental Accounting System (DAS) 
because both TPMS and TFMS interface 
with it (Exhibit 20.5). 


20.50 Treaties with First Nations date 
back to the 18th century; funding 
arrangements span many years. 
Consequently, application systems like 
TPMS and TFMS were developed using 
an eight-digit date code. Indian and 
Northern Affairs Canada determined that 
some conversion work was needed for 
data display and reporting. Independent of 
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Exhibit 20.5 


Indian and Northern Affairs 
Canada — First Nations 
Transfer Payments and Trust 
Funds 


its Year 2000 work, the Department was 
also changing its operating system and 
data base standard. The change involved a 
conversion of the Departmental 
Accounting System (DAS), which was 
considered Year 2000 compliant by the 
supplier. After conversion, the system 
would need to be tested using the new 
technology platform to ensure that it 
would be Year 2000 compliant. 


20.51 We found that the Department 
had not addressed the Year 2000 problem 
on a department-wide, business-oriented 
basis. Under the direction of its 
Information Management Branch, the 
Department’s Year 2000 project activities 
at the time of our audit focussed primarily 
on managing changes in the areas of its 
corporate information systems and 
technology. Only at the end of June 1998 
did the Department formally appoint a 
senior sponsor to direct its Year 2000 
project and take on a department-wide 
perspective. 


20.52 Indian and Northern Affairs 
Canada prepared a master list of software 
systems — an inventory list of corporate 


systems — during the planning phase of 
its Year 2000 project. However, we found 
that the list did not identify all the 
significant systems that interface with 
those in the inventory list. The Resource 
Management Information System (RIMS) 
was omitted — a local system that serves 
as the primary source of data input to the 
Trust Fund Management System (TFMS). 
In addition, the list of interfaces for DAS 
did not include the linkage with the 
government’s payment system or the 
central accounting system. As a result of 
our audit observation, the Department has 
added RIMS and those linkages to its 
inventory of Year 2000 work. 


20.53 We found that work plans have 
been prepared for all three mission-critical 
systems. However, they do not contain 
details of the activities to be performed 
and the corresponding resources needed. 
Thus, it will be difficult for the 
Department to monitor progress and assess 
the adequacy of the estimated resource 
requirement. We also noted that test plans 
for the systems have not been prepared. 


20.54 In our view, the Year 2000 risks 
to TPMS, TFMS and DAS are lower than 


Systems selected for audit 


e Transfer Payment Management System (TPMS) 
e Trust Fund Management System (TFMS) 


e Departmental Accounting System (DAS) 


Relationship to 
mission-critical function 
that they support 


The TPMS manages and provides funding of $3 billion 
annually to over 600 First Nations. The TFMS accounts for 
trust funds of about $1 billion that the Department manages on 
behalf of First Nations. Both systems interface with DAS, the 
accounting system for the Department. 


Systems description 


All three are customized systems. TPMS and TFMS were 
developed in the 1980s, with 600 modules and about 260 
modules respectively; DAS was redeveloped using PowerHouse 
and C language and has about 50,000 lines of code. 


Status (30 June 1998) 


follow. 


The technology environment was being upgraded. The TPMS 
and TFMS were in conversion and a Year 2000 compliant 
version of software has been installed for DAS, with testing to 


Testing plans and strategy 
plans 


Testing strategy exists for the three systems but not detailed 


Planned completion dates 


31 December 1998 
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the risks to some of the other 
mission-critical systems we examined. In 
comparison with others, these systems are 
less complex. When we completed our 
audit, the Department had six months 
remaining to complete its conversion work 
and test the systems by its target date of 
December 1998. Both TPMS and TFMS 
now use an eight-digit date code and DAS 
has been upgraded to a version that is 
considered Year 2000 compliant. 


20.55 At the end of our audit, Indian 
and Northern Affairs Canada was planning 
to complete a risk assessment by 

31 August 1998 for submission to the 
Secretariat. According to the Department, 
preparation of an information technology 
contingency plan and a business 
contingency plan would follow. 


Revenue Canada — border crossing 
services for commercial goods 


20.56 Revenue Canada’s customs 
service administers over 80 acts and 
regulations on the government’s behalf at 
border crossings. For commercial goods, 
the responsibilities vary from facilitating 
shipment and import of commercial goods 
to intercepting illicit drugs and other 
banned substances. Customs processes 
over 11 million commercial entries 
annually, accounting for about $20 billion 
in customs duties and the goods and 
services tax. It is a mission-critical 
function that not only provides an 
essential service to importers but can also 
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affect public health and safety and 
Canada’s finances. 


20.57 The Customs Commercial 
System (CCS) operates on a continuous 
basis. It maintains an inventory of all 
commercial entries and directly supports 
the processing of and accounting for the 
goods. Further, it interfaces with the 
Accelerated Commercial Release 
Operating Support System (ACROSS), a 
departmental system that supports customs 
officers in making decisions on the release 
of goods (Exhibit 20.6). 


20.58 The Year 2000 conversion 
strategy for CCS involved repairing 
existing codes. The Department decided to 
implement the repaired codes in a single 
release. 


20.59 Over the years, in meeting the 
need to update the tax code annually, 
Revenue Canada has developed a 
‘structured methodology for testing code 
changes. In preparing for the testing phase 
of its Year 2000 project, it augmented its 
regular testing process with a Year 2000 
testing environment (called LPAR). This 
initial testing strategy was reviewed by an 
information technology firm, which 
recommended adding a test environment 
called WT. The WT environment is 
capable of automated date manipulation to 
age data. It can help identify major 
problems in an application before the 
Department tests it in its LPAR 
environment. 


System selected for audit 


Customs Commercial System (CCS) 


Exhibit 20.6 


Relationship to 
mission-critical function 
that it supports 
those goods. 


CCS is a key system that supports services at border crossings 
for the import of commercial goods. Its functions include 
keeping inventory, processing, accounting for and control of 


Revenue Canada — Border 
Crossing Services for 
Commercial Goods 


System description 


CCS consists of 4,283 modules with about 1.6 million lines of 
| code. It operates in a mainframe MVS/DB2 environment. 


Status (30 June 1998) 


Testing plans and strategy 


Planned completion dates 


Conversion phase for CCS had been completed. The system 
was in integration testing and testing for user acceptance. 


Testing plans and strategy were completed. 


30 October 1998 
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20.60 The recommendation was 
accepted, bringing the testing process for 
Year 2000 to four phases of regular testing 
as well as testing in the two Year 2000 
environments. Revenue Canada has a 
department-wide definition of Year 2000 
compliance. Systems are certified for 
Year 2000 only if they have met the 
compliance definition, tested successfully 
for user acceptance and in the LPAR 
environment, and been reintroduced in the 
normal operating environment. 


20.61 We found that repair work on 
CCS had been completed, and the system 
was in testing at the time of our audit. 
Testing in the LPAR environment was 
scheduled for October 1998. 


20.62 We note that there has been some 
slippage in the deadlines for testing and 
that the time allowed for the testing 
phases has been compressed. In particular, 
as of 30 June 1998, the WT integration 
testing had not been completed and the 
objective of conducting WT testing — to 
identify and correct major problems early 
in the process — had not been achieved. 


20.63 Revenue Canada’s Information 
Technology Branch noted in February 
1998 that testing was becoming the 
bottleneck for Year 2000 projects in 
industry as well as in the Department. It 
stated that its LPAR testing schedules 
were not negotiable. As a result, at the 
time of our audit there was a risk that the 
CCS might not undergo sufficient testing 
prior to being tested in the LPAR 
environment in October 1998 and that 
problems could arise during that critical 
phase. Thus, there was a risk that CCS 
would not achieve Year 2000 certification 
by the planned date. 


20.64 We noted that two releases of 
CCS have been planned for 1999; the first 
is scheduled for January. In the event that 
CCS failed to achieve Year 2000 
certification in October 1998, the 
Department was planning to certify the 
system in early 1999. 


20.65 Although some risks exist for 
CCS, in our view they are lower than the 
risks to some of the other systems we 
examined in this audit. If CCS failed to 
achieve Year 2000 certification in October 
1998, it would have to be given priority 
for LPAR testing in 1999 and the 
Department would need to consider 
undertaking risk assessment and 
contingency planning. Subsequent to the 
audit, the Department advised us that its 
certification testing for CCS was near 
completion and the system was on 
schedule to be implemented in October 
1998 as planned. 


Royal Canadian Mounted Police — law 
enforcement services 


20.66 Law enforcement services 
support law and order and provide for the 
safety and security of the public. Thus, 
they form part of the government’s 
mission-critical functions. The Canadian 
Police Information Centre (CPIC), a 
system that supports policing service 
across Canada, is maintained by the Royal 
Canadian Mounted Police (RCMP) and is 
used by its 700 detachments and over 
500 other agencies (Exhibit 20.7). 


20.67 The Year 2000 strategy for CPIC 
involved two phases: 


e converting the application and 
operating systems from COBOL to MVS 
COBOL; and 


e repairing application systems using a 
four-digit year code. 


20.68 Many application systems use 
forward dates and can fail or malfunction 
prior to 2000. The CPIC modules that 
could be expected to fail in January 1999 
have been identified. Since the conversion 
strategy involves repairing codes module 
by module, those with a potential early 
failure date have been given priority. 


20.69 We found that the RCMP has an 
overall Year 2000 project plan for the 
entire CPIC system. Activities in that plan 
range from one week to many months in 
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duration. In the absence of detailed plans, 
it is difficult for the Year 2000 project 
office to assess whether scheduled dates 
for task completion are reasonable, and to 
monitor progress. In these circumstances, 
any delays could go undetected for 
months. 


20.70 As of June 1998, there had been 
only minor slippage in the first phase of 
converting to MVS COBOL. But phase 
two, year code conversion, had 
experienced a three-month delay. 
Although the overall project plan had been 
updated accordingly, which changed the 
target date for full CPIC compliance from 
31 December 1998 to 31 March 1999, 
target dates for some key activities had 
not yet been set. 


20.71 The RCMP has established a 
Year 2000 test environment (called 
LPAR), and some guidelines exist on 
testing applications for specific dates. A 
Year 2000 test plan and a test strategy, 
including co-ordinated testing with 
external users, were still being developed 
at the time of our audit. 


20.72 The CPIC system is used 
extensively by the RCMP and other 
organizations involved in law 
enforcement. Outside users who have data 
interfaces with CPIC are categorized as 
follows: 


e Category 1, police departments; 
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e Category 2, departments and 
agencies with some law enforcement 
roles; and 


e Category 3, departments and 
agencies with limited law enforcement 
roles. 


Category | users are the primary users 
who exchange high volumes of data with 
CPIC. Data interface between the RCMP 
and the Federal Bureau of Investigation 
(FBI) and other law enforcement agencies 
in the United States is provided through 
the Automated Canadian United States 
Police Information Exchange System 
(ACUPIES). 


20.73. The RCMP has identified all data 
interface partners in its inventory phase 
for Year 2000. In January 1998, it 
communicated with all Category 1 users 
to advise them of its plans and the date 
when CPIC modules would start using a 


four-digit year code. It has also 


communicated with the FBI and both 
parties have agreed to have their 
respective systems accept a two-digit and 
a four-digit year code from January until 
June 1999, when they expect both systems 
to be ready. 


20.74 However, as of 30 June 1998, the 
response from Category | users had been 
limited. As a result, the RCMP had little 
information on whether its plans or time 
frame are acceptable to those users. 


20.75 The prime use of the CPIC 
system is to provide information to law 


System selected for audit 


Canadian Police Information Centre (CPIC) system 


Relationship to 
mission-critical function 


CPIC maintains information and provides it on demand to 
police officers in support of law enforcement activities in crime 


that it supports prevention, investigation and pursuit of criminals. 
i + - - 

System description CPIC was designed about 25 years ago. It has approximately 
750,000 lines of code and operates in a mainframe COBOL 
environment. 

Status (30 June 1998) Conversion to MVS COBOL was in testing; conversion of 
application systems and modules had started. 

Testing plans and strategy —_The testing plans and strategy were being developed. 


Planned completion dates | 


i 


31 March 1999 


Exhibit 20.7 


Royal Canadian Mounted Police 
— Law Enforcement Services 
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The RCMP has noted 
that contingency 
planning is actively 
under way, with a very 
comprehensive 
checklist that will be 
used at every 


enforcement personnel on demand. Thus, 
the ability to continue to exchange data 
and provide information is of paramount 
importance. In our view, follow-up with 
Category | users is required and, further, 
the RCMP needs to extend its 
communication to users in categories 2 
and 3. Subsequent to the audit, the RCMP 
advised us that it had started to follow up 
with its Category 1 users and it intended 
to communicate with users in the other 
two categories. 


20.76 We also noted a Year 2000 
project risk associated with the 
availability of technical resources. The 
lack of technical resources was a primary 
cause of the delays noted in paragraph 
20.70. We found the resource situation in 
the application development and the 
quality control and testing areas to be of 
particular concern. For example, at the 
time of the audit only one individual had 
been identified to conduct testing for the 
entire CPIC system. 


Progress Monitoring by Treasury 
Board Secretariat 


20.79 In May 1996, the Treasury Board 
Secretariat established a project office 
within its Chief Information Officer 
Branch to provide leadership for the Year 
2000 issue in government. The project 
office has been charged with the 
responsibility to oversee Year 2000 efforts 
in the government and to address common 
needs of departments and agencies. In our 
1997 audit, we reported that the 
Secretariat had helped to raise awareness 
across the government and had facilitated 
the exchange of experiences and views on 
Year 2000 efforts. 


20.80 In response to our 1997 
recommendations, the Secretariat stated 
that it had started to examine the state of 
readiness of all systems that the 
government considers mission-critical, in 
order to confirm that they would continue 
to function properly after 31 December 
1999; 


Survey results for systems supporting 
government-wide mission-critical 


detachment across the 20.77. We have concluded that the CPIC : 
functions 
system continues to be at risk. In 
country. particular, if end-to-end testing and 20.81 The project office conducted a 
co-ordination with the user community are survey of 76 departments and agencies in 
not addressed adequately, the use of the the fall of 1997. It tabulated separately the 
CPIC system could be greatly curtailed. survey results in the 19 departments 
responsible for systems that support 
. government-wide mission-critical 
20.78 At the WEES of our audit, the functions, in 16 other large departments 
RCMP was planning to prepare a Year and agencies, and in the remaining 41 
2000 risk assessment by 31 August 1998 smaller departments and agencies. 
and submit it to the Treasury Board ; 
Secretariat. The RCMP has indicated that 29-82 _In its report of January 1998, the 
it has a continuous risk management Secretariat noted that although almost all 
process in place using outside consultants departments and agencies had made 
to assure an independent and objective significant prog only a few had 
review, and has made continuous and carried out substantial repair work or 
significant progress. It has also noted that testing of systems by the fall of 1997. It 
contingency planning is actively under concluded that there was sufficient time to 
way with a very comprehensive checklist complete the conversion of 
that will be used at every detachment mission-critical systems but that some 
across the country, and that it has played a non-essential systems might not be 
lead role in educating many police forces converted at the same pace. 
and municipalities across Canada on the 20.83 The project office conducted a 
issue of Year 2000. further survey in the spring of 1998, 
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focussing primarily on the mission-critical 
functions of the government. Those results 
were updated in June 1998 and the project 
office has monitored progress on a 
monthly basis since then. 


20.84 In order to assess progress, the 
Secretariat used a model from a major 
information technology research firm. The 
firm assigns to each of its seven phases of 
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According to the targets, departments 
and agencies should have completed 
76 percent of their Year 2000 work by 
30 June 1998. 


20.86 As of 30 June 1998, however, the 
Secretariat’s summary results showed that 
overall, the 21 departments and agencies 
had completed about 50 percent of their 
Year 2000 efforts for systems that support 


According to the 


targets, departments 
and agencies should 


a Year 2000 project an estimate of level of aS AMES RG: ae cy have completed 
PER § ncngeh ypc be nas Anco ck functions. Two Crown corporations f F 
eo P ee responsible for two such functions were 76 percent of their 

Ser ie OMe ee is added recently and information on their Year 2000 work b 
generic model we used in 1997 with those 5 y 
in the firm’s model; the firm’s estimate of Se ee eae grey pa 30 June 1998 
level of effort is shown as a percentage. EE SSRN ASE CRA Oe 

that the pace of Year 2000 action had 

accelerated in recent months. 
20.85 In the fall of 1997, the research 
firm considered that an organization 20.87 In the spring of 1998, the 
should by then have completed 45 percent Secretariat announced that the target date 
of its Year 2000 efforts in order to have for full implementation of all systems that 
fully compliant systems in place by support mission-critical functions was 
31 December 1999. On that premise, the summer 1999, later confirmed as June 
Secretariat set targets for percentage ~ 1999 and subsequently endorsed by 
completion based on elapsed time. Treasury Board ministers. Given that this 

Exhibit 20.8 


Year 2000 Project Model and Percentage Level of Effort 


Auditor General Model Used by 
Year 2000 Project Model Treasury Board Secretariat 
ae Inventory, Assess the Year 2000 impact on the Awareness 
Assessment department. Inventory and assess systems Inventory 
Project Management & Planning supporting the major programs and Project Scoping 
Structure operations and prioritize their conversion. | Analysis and Design 
Formulate an overall strategy and action | (26%) 
Establish a Year 2000 plan. Develop contingency plans. 
program management t = = 
structure. Promote Conversion Repair, replace or retire selected Repair 
department-wide technology, systems and data bases. Modify (20%) 
awareness. Secure interfaces. 
necessary resources and =e = 
management support. Testing & Validate repaired or replaced systems and Testing 
Monitor progress, Implementation interfaces for Year 2000 compliance. Test Implementation 
Sustain management the performance, functionality and (54%) 
commitment and integration of technology, systems, data 
support. bases and interfaces in an operational 
environment. Implement repaired or 
replaced systems and technology. || 


| Implement contingency plans and 


work-around arrangements as necessary. | 
Li 


Sources: 1997 Report of the Auditor General, Chapter 12; Treasury Board Secretariat 
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Exhibit 20.9 
Year 2000 Progress of Systems 


Supporting Government-Wide 
Mission-Critical Functions 


Source: Treasury Board Secretariat 


extends the original model by six months, 
and assuming the same pace of action as 
seen in the spring of 1998, the Secretariat 
takes the position that completion by June 
1999 is achievable (Exhibit 20.9). 


20.88 | However, in adapting the 
research firm’s model, the Secretariat had 
set its target using elapsed time as the 
main proxy for level of effort. Its 
indicator, percentage completed, did not 
take into account other measures such as 
percentage of staff hours used or quantity 
of work done in comparison with the 
overall budget. The quantity of work done 
is often measured in lines of programming 
code. For some departments and agencies, 
the Secretariat used the number of lines of 
programming code as a weighting factor 
in assessing progress. 


20.89 In addition, the model was 
intended for situations that primarily 
involved repairing and testing existing 
systems. The methodology may not be as 
applicable by departments that have 
decided as their conversion strategy to 
replace non-compliant systems. In those 
cases, progress reporting based on elapsed 
time may not always be meaningful. In 
the Secretariat’s view, the model is 
adaptable to situations involving the 
replacement of systems. However, as of 
June 1998, it had not adapted the model to 


Percentage Completion 


100 


that use in assessing the progress of 
departments and agencies. 


20.90 In 1997, we reported that the 
testing and implementation phase took 
more effort than all other phases 
combined. It is now widely acknowledged 
in the information technology industry 
that the testing phase of Year 2000 
projects generally has proved even more 
time-consuming than anticipated. Thus, a 
simple linear projection of percentage 
completion against a target elapsed time 
can prove to be optimistic. 


20.91 The Secretariat did not validate 
data and information that it collected in its 
survey. Although some departments and 
agencies used internal audit or consultants 
to validate their Year 2000 progress, 
others did not. Best practices in industry 
have Year 2000 project offices seeking 
independent assurance to validate progress 
reports that they have received from units 
in their organizations. Some project 
offices go to the extent of using automated 
tools to verify the converted programming 
codes. Experience in some private sector 
firms showed that upon validation the 
reported percentage completed was often 
found to be overstated. 


20.92 We compared our findings on the 
systems that support the six mission- 
critical functions we audited with 
information that the same departments and 


80 | TBS Model 

60 

40 

20 
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agencies had provided to the Secretariat. 
In most cases, the start dates of work and 
the completion dates that we had noted in 
the departments were consistent with 
those in the Secretariat’s files. In general, 
its files did not contain sufficient details to 
use the number of lines of programming 
code in support of analysis of quantity of 
work done and assessment of progress 
toward making systems Year 2000 
compliant. 


20.93 | While the Secretariat’s survey 
asked for information on data interfaces 
and embedded systems, these were not 
used in the calculation of percentage 
completed. Consequently, the assessments 
of departments and agencies with 
significant data dependencies or numerous 
embedded devices may not adequately 
reflect their Year 2000 progress. 


20.94 We reported last year that 

31 December 1998 was emerging as the 
norm in the information technology 
industry for full implementation of 
compliant systems. We are concerned that 
the Secretariat’s target date for full 
implementation allows only six months to 
address slippage and unexpected 
problems. Further, some systems 
supporting the 48 government-wide 
mission-critical functions have target 
dates beyond June 1999 for full 
implementation. Most of the systems we 
examined this year have experienced 
delays, some by up to four months. 


Many mission-critical systems remain at 
risk 


20.95 An overall rate of completion for 
systems supporting government-wide 
mission-critical functions can provide no 
more than a general indication of Year 
2000 progress. Given that the average rate 
of completion was 50 percent at 30 June 
1998, some departments and agencies 
were progressing at below that rate. 


20.96 According to the Secretariat’s 
summary results at the end of June, 
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12 departments and agencies responsible 
for 28 mission-critical functions had a 
more than 50 percent completion rate for 
applications, systems and devices 
supporting those functions. The 
Secretariat also noted that information for 
two departments was not complete and 
that this could affect their reported 
percentage completion. 


20.97 Overall, the Secretariat’s 
summary of the progress made at 30 June 
1998 by the 21 departments and agencies 
with mission-critical functions was as 
follows: 


e 4 departments and agencies with 
good progress (over 60 percent complete); 


e 10 departments and agencies with 
satisfactory progress (46 to 60 percent 
complete); and 


e 7 departments and agencies with 
difficulties (45 percent complete or less). 


Progress information on the two newly 
added Crown corporations was not 
available in June 1998 and they were not 
included in the summary results. 


20.98 Although the Secretariat’s 
monitoring of progress showed that Year 
2000 action in the government had 
accelerated, as of 30 June 1998 only four 
departments and agencies, responsible for 
7 of the 48 mission-critical functions, 
were rated as having made good progress; 
work on 18 functions in nine departments 
and agencies had a score of 50 percent 
complete or less. 


20.99 Weare very concerned that as of 
30 June 1998, various of these 
government systems supporting 
mission-critical functions remained at risk 
of not being ready in time. 


20.100 In August 1998, the Secretariat 
advised us that some of its analysts had 
started attending Year 2000 project 
meetings in the 21 departments and 
agencies and the two Crown corporations. 
In addition, its Year 2000 project director 


As of 30 June 1998 
only four departments 
and agencies, 
responsible for 7 of 
the 48 mission-critical 
functions, were rated 
as having made good 
progress. 
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had met with senior staff in some 
departments to discuss their progress. 


20.101 The Secretariat had also written 
to departments and agencies responsible 
for systems that support the mission- 
critical functions to request that risk 
assessments be completed by 31 August 
1998. For systems considered to be at risk 
of not meeting the implementation target 
date, detailed contingency plans will be 
requested by 31 December 1998. 


20.102 In our 1997 audit chapter, we 
recommended that the Secretariat 
intervene strategically as appropriate to 
ensure the successful implementation of 
the government’s mission-critical systems. 
Near the end of this year’s audit, although 
it had held some meetings with certain 
departments to discuss Year 2000 
progress, there were no documented plans 
to indicate what would trigger 
intervention, at what dates, and what form 
it would take. 


20.103 Our 1997 chapter also raised the 
issue of triage, which provides for the 
redeployment of resources to those 
systems that are most critical to an 
organization. While the Secretariat 
acknowledged last year that triage may 
become necessary at some point, to date it 
is not evident what criteria would be used 
to judge this and how resources could be 
redeployed among departments and 
agencies. 


20.104 If systems that support 
mission-critical functions were to remain 
at risk, it would be essential that 
contingency plans be developed and tested 
prior to 2000. Overall, the government’s 
objective for Year 2000 readiness is to 
ensure that essential programs and 
services continue for its beneficiaries and 
stakeholders. The ideal would be fully 
compliant systems that would support 
mission-critical functions; interim 
work-around arrangements may be 
acceptable for the short term. 


20.105 The government should 
continue to rank Year 2000 as a top 
priority and further accelerate its 
efforts on those systems that support its 
mission-critical functions, to reduce the 
risk that they may not be fully 
compliant before 2000. 


20.106 The Treasury Board Secretariat 
should determine when and how it 
ought to exercise strategic intervention 
to ensure Year 2000 compliance of 
systems that support mission-critical 
functions in government. It should also 
develop plans for triage and put them in 
place as necessary. 


20.107 Where systems that support 
mission-critical functions remain at 
risk, the Secretariat should ensure that 
contingency plans are developed and 
tested prior to 2000 so that essential 
government programs and services 
continue after 31 December 1999. 


20.108 The Secretariat should examine 
ways to further improve the quality of 
its survey data and information, 
including the use of other measures in 
analyzing and assessing progress and 
the independent verification of progress 
information. 


Treasury Board Secretariat’s response: 
The Treasury Board Secretariat agrees 
with and supports the main points of the 
chapter. The chapter highlights areas of 
both concern and progress, areas with 
which we have widely been reported to 
agree. 


We feel that it is critical to note that 
progress is an ongoing fact during the 
Year 2000 remediation programs in the 
departments and agencies we are 
monitoring. Not only is the percentage of 
completion (as reported monthly) steadily 
increasing, but the pace of change is also 
increasing. In fact, since the audit was 
completed at the end of June 1998, the 
overall government readiness has moved 


from 52 to 64 percent. 


We agree with the audit recommendations 
that the government continue to rank Year 
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2000 readiness as a top priority. The 
Treasury Board will continue to receive 
regular monthly reports on progress. Year 
2000 will also be the topic of 
correspondence from the President of the 
Treasury Board to his Cabinet colleagues, 
and from the Secretary to deputy 
ministers. 


The rate of compliance continues to 
improve; nonetheless, we agree that 
prudent project management 
acknowledges the fact that slippage can 
occur, and triage may have to occur. The 
Treasury Board Secretariat will be 
working with departments and agencies in 
the development of plans for the triage of 
systems that support government-wide 
mission-critical functions. 


The Secretariat is working with 
departments and agencies to assist, where 
possible, in the development or refinement 
of contingency plans to ensure that 
government-wide mission-critical 
functions continue. 


The Secretariat does not rely exclusively 
on elapsed time as an indicator of 
progress. It is willing to consider changes 
to improve the quality of reporting, 
provided that work presently being done 
on remediation is not jeopardized. We 
have recommended to departments and 
agencies the engagement of their own 
internal audit staff to provide independent 
verification and validation of the results 
our analysts collect and report. 


Initiatives to Address Common 
Needs 


Some progress made in addressing 
exposure for Year 2000 


20.109 In the 1997 audit, we identified a 
number of areas in which Year 2000 
projects faced significant exposure to risk: 


* competing priorities and 
developments; 


e« insufficient technical resources; 
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e failure or delay in obtaining 
compliant upgrades from vendors; 


e data interface exposure; and 
e risk of funding delay. 


In the audit report, we also noted that 
there were opportunities for efficiency 
gains by addressing common issues 
horizontally. In the current audit, we 
observed that progress had been made in 
various areas. 


20.110 For example, the Secretariat 
wrote on several occasions to deputy 
heads and heads of agencies to emphasize 
the significance of the Year 2000 threat 
and the high priority that it ought to be 
assigned. In March 1998, the government 
also directed departments and agencies to 
consider Year 2000 implications in all 
policy proposals. 


20.111 We noted in 1997 that 
departments were expressing concerns 
about availability of technical resources; 
many were experiencing higher than 
normal attrition rates among computer 
specialists. Since that time, efforts have 
been under way to identify and attract 
information technology professionals into 
government. The conclusion of contract 
negotiations with the labour union for the 
government’s computer specialist 
community has helped to retain staff. In 
addition, an omnibus contract has been 
awarded to seven private sector firms to 
support Year 2000 remediation and testing 
work on systems that support 
government-wide mission-critical 
functions. 


As of July 1998, the 
Secretariat had 
approved loans 
totalling about 
$365 million to 15 
departments and 
agencies. 


20.112 In 1997, the Secretariat estimated 
the government’s total costs for Year 2000 
work at $1 billion. During our 1998 audit, 
its Year 2000 project office advised us 
that the cost estimates had been revised 

to $1.4 billion. To address the risk of 
delay in some Year 2000 projects in the 
event that funds could not be secured on 

a timely basis, the Secretariat made 
provisions to make up to a total of 

$400 million available for loans to support 
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Embedded systems 
and devices are 
emerging as a 
significant concern. 


the departments and agencies responsible 
for mission-critical functions. As of 

July 1998, the Secretariat had approved 
loans totalling about $365 million to 

15 departments and agencies. 


Work plan needed for remaining 
horizontal issues 


20.113 While some progress has been 
made in addressing common needs, 
exposure remains in the areas we have 
noted. There is also significant exposure 
in other areas. 


20.114 For example, the availability of 
technical resources continues to be an area 
of significant exposure for all Year 2000 
projects. Although departments indicate 
that attrition rates have stabilized since the 
conclusion of contract negotiations for the 
government’s computer specialists, staff 
turnover has continued. In examining 
mission-critical systems, we noted a 
specific case of slippage for which 
shortage of technical resources was cited 
as the reason. Further, the contract for 
computer specialists in government will 
expire in 1999, when Year 2000 work will 
still be critical. 


20.115 Data interface also represents an 
area of significant exposure. Information 
technology research firms have 
emphasized the need for end-to-end 
testing among organizations that interact 
electronically. In describing its model for 
comparing Year 2000 progress among 
organizations, one research firm indicated 
that such testing would be 
time-consuming and in some cases 
difficult to accomplish. 


20.116 Legal liability is another area of 
exposure. In response to our 1997 
recommendations, the government 
assigned a lawyer to co-ordinate its legal 
efforts. Subsequently, legal co-ordinators 
were identified in various departments and 
agencies. The efforts helped to raise Year 
2000 awareness in the government’s legal 
community and resulted in a Year 2000 


issue paper. In March 1998, the issue 
paper was released to departments and 
agencies to help them in assessing the 
legal risks to their respective organizations 
as aresult of Year 2000. Nevertheless, 
given that the government has many 
regulatory responsibilities, the 
implications of the Year 2000 threat have 
yet to be fully analyzed and addressed. 


20.117 Embedded systems and devices 
are emerging as a significant concern. 
Numbering in the billions, they perform a 
vast range of duties serving many domains 
such as science, engineering, 
manufacturing and health care. Experts 
have estimated that only a small 
percentage of these micro processors are 
date-sensitive. Even so, an estimated 

20 million to 250 million could fail as a 
result of the millennium bug. 


20.118 In January 1998, the Treasury 
Board Secretariat tasked a small team of 
assistant deputy ministers to review Year 
2000 readiness in government. In 
February 1998, the team reported the 
issues and areas of exposure we have 
noted, as well as other impediments to 
success. It concluded that the Year 2000 
challenge could be overcome only if 
decisions and actions were taken to 
address the issues it had identified. 


20.119 As of August 1998, the 
Secretariat had started to define broad 
roles for some departments and agencies 
to address a number of those issues. In 
early October we were advised by the 
Secretariat that it had identified two 
initiatives to assist departments and 
agencies in dealing with embedded 
technology, and that it planned to hold a 
symposium on embedded devices and 
technology later that month. The 
Secretariat indicated that the initiatives 
would proceed once funding became 
available. However, a work plan that 
includes the following has yet to be 
developed: 


e a full list of the issues to be 
addressed; 


20-26 


Report of the Auditor General of Canada — December 1998 


e roles and responsibilities of each 
organization assigned to address them; 


e the resources required; 
e the desired deliverables; and 
e atime frame for their completion. 


As we reported in 1997, if the issues are 
not pursued expeditiously, the potential 
benefits of the horizontal effort to help 
departments and agencies overcome the 
Year 2000 threat will not be fully realized. 


20.120 The Treasury Board Secretariat 
should develop a work plan and take 
steps to facilitate the resolution of 
common horizontal issues that 
departments and agencies face in 
addressing Year 2000. 


Treasury Board Secretariat’s response: 
Plans and efforts have been put into place 
for monitoring and assessing departmental 
readiness, and identifying horizontal 
readiness issues through interdepartmental 
working groups and other means. With the 
need for a more frequent and detailed 
reporting schedule to Treasury Board 
ministers, and for addressing overall 
issues related to government readiness, 
Treasury Board Secretariat agrees that 
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the potential impact of Year 2000 on the 
government and about the government’s 
progress in making its systems compliant. 


20.122 We found, in this audit, that the 
government has kept ministers informed 
about the priority and significance of Year 
2000. Further, since our 1997 audit the 
Secretariat has appeared several times 
before the Treasury Board to discuss the 
Year 2000 issue and provide information 
on the government’s progress, and to seek 
approval for specific measures and related 
funding as necessary. As we neared the 
completion of our audit work in 1998, the 
Secretariat advised us that it would be 
providing Treasury Board with monthly 
updates of Year 2000 progress. 


Information to Parliament needs to be 
improved 


20.123 Subsequent to the tabling of our 
October 1997 Report to Parliament, two 
standing committees of the House of 
Commons held hearings on the subject of 
Year 2000. The Standing Committee on 
Industry studied industry readiness; the 
Standing Committee on Public Accounts 
focussed on Chapter 12 of our Report, on 
the government’s preparedness for Year 
2000. The Chief Information Officer of 


The information that 
the Public Accounts 
Committee had on 


additional and more precise plans must be __ the Treasury Board Secretariat appeared Year 2000 progress 
prepared. A master plan for reporting to as a witness at hearings of both was almost one year 
Ugh has already been completed and committees. out of date. 
the integrating of additional activities is 20.124 In November 1997 the Public 
iil Accounts Committee tabled its Second 
Reporting Information to Ministers Gye ior ict our Repo. 
and Parliament recommended, among other things, that 
the Treasury Board Secretariat prepare 
Information on Year 2000 urgency and semi-annual reports on Year 2000 progress 
progress has been provided to ministers —_ and submit them to the Committee, 
20.121 In 1997, we recommended that ee eM LE 
departments and agencies work to engage 20.125 The government accepted the 
the involvement and concern of ministers recommendations and the Secretariat 
and advise them of the urgency of the submitted its first progress report to the 
Year 2000 threat. We also recommended Committee in early February 1998. The 
that the Secretariat advise the Treasury progress report was dated January 1998 
Board periodically of the government’s and reflected status as of the fall of 1997. 
progress. In addition, we recommended As of 31 August 1998, no further progress 
that Parliament be kept informed about reports had been submitted. Consequently, 
Report of the Auditor General of Canada — December 1998 20-27 


Preparedness for Year 2000: Government-Wide Mission-Critical Systems 


the information that the Public Accounts 
Committee had on Year 2000 progress was 
almost one year out of date. The 
Secretariat has since advised us that in late 
September 1998 it submitted a second 
progress report to the Committee. 


20.126 Although the Committee also 
recommended that information on 
departments’ Year 2000 progress be 
included in their Performance Reports, we 
found no evidence that the Secretariat had 
communicated this to departments. Four 
departments and agencies that we audited 
this year referred to Year 2000 in their 
1998-99 Estimates documents and only 
one department reported on the status of 
its Year 2000 project in its 1997 
Performance Report to Parliament. 


20.127 The government should 
improve its reporting of information to 
Parliament on its progress in making 
systems compliant for Year 2000. 


Treasury Board Secretariat’s response: 
Treasury Board Secretariat has reported to 
the Public Accounts Committee, the 
Industry Committee, and to Cabinet. We 
will continue to examine ways of keeping 
Parliament informed. 


Conclusion 


20.128 One year after our 1997 audit, we 
found that the government has given the 
Year 2000 threat a high priority and 
accelerated the pace of its work on Year 
2000 projects. Progress has also been 
made in monitoring Year 2000 work on 
government-wide mission-critical 
functions and in addressing certain 


common issues that affect many 
departments and agencies. 


20.129 However, in reviewing the June 
1998 survey results summarized by the 
Treasury Board Secretariat, we observed 
that only four departments responsible for 
7 of the 48 mission-critical functions in 
the government were rated as having 
made good progress; nine departments and 
agencies responsible for 18 mission- 
critical functions had a score for work 
completed of 50 percent or less. The 
results of our examination also showed 
that, as of June 1998, several key systems 
supporting three of six government-wide 
mission-critical functions remained 
exposed to the Year 2000 threat. 


20.130 We have concluded that with 

18 months left, many systems that support 
mission-critical functions remain at risk of 
not being ready before 2000. We are very 
concerned that some essential government 
services could be interrupted at the start 
of 2000. 


20.131 Notwithstanding the accelerated 
pace of work and the progress made since 
our last audit, more needs to be done. 
There needs to be a continued focus on 
Year 2000, including strategic intervention 
by the Treasury Board Secretariat as 
appropriate, and preparation and testing of 
contingency plans by departments and 
agencies to ensure that those 
mission-critical functions continue 
without interruption into 2000. 


20.132 As we reported in 1997, other 
areas remain exposed to risk and merit 
common, horizontal efforts to address 
them. Reporting of information to 
Parliament on Year 2000 issues needs to 
be improved. 
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% About the Audit 


Objective and Scope 


The 1998 audit focussed on selected government-wide mission-critical functions and selected systems that 
support them. Our objective was to assess the progress the government has made in identifying and mitigating 
the risks to its systems that support mission-critical functions as a result of the millennium date code problem, 
and to report on this to Parliament. 


We examined actions taken by the government to address our October 1997 Report recommendations. We | 
obtained an update of the government’s state of readiness for Year 2000 at 30 June 1998 by reviewing survey 
results collected by the Treasury Board Secretariat. We have not audited the representations made by | 
departments and agencies in the surveys. | 

| 


In addition, we assessed Year 2000 risks to certain systems and applications that support six mission-critical 
functions of the government. The functions were selected on the basis of their significance to health and 

safety, to financial and legal implications and to essential services and support for program beneficiaries. | 
Those systems and applications operate in the following departments and agencies: 


e Canadian Food Inspection Agency; - | 
e National Defence; 

e Human Resources Development Canada; 
e Indian and Northern Affairs Canada; 

e Revenue Canada; and 

e Royal Canadian Mounted Police. 

The departments of National Defence, Human Resources Development Canada and Revenue Canada were 


among the nine departments that we audited in 1997. 


Our findings pertain only to the systems and applications that we examined and are not representative of 
general progress in those departments and agencies. 


Criteria 


The general criteria used in this audit followed those used in our 1997 audit of the government’s overall state 
of Year 2000 preparedness: 


e There should be ongoing leadership initiatives to sustain commitment and support from departmental 
and/or agency senior management. 


e Systems and applications that are critical to government as a whole should be identified, and the progress 
of their Year 2000 conversion, testing and implementation should be monitored and reported periodically. 
Where necessary, strategic intervention should be exercised to ensure that these systems and applications 
are successfully implemented before 2000. 
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e Where risk of failure remains a threat to critical functions of government, contingency plans should be 
developed and measures put in place to provide for appropriate transition to 2000. 


e Initiatives should be in place to co-ordinate and facilitate Year 2000 efforts of departments and agencies 
to maximize their cost effectiveness. 


e Parliament should be kept informed of matters of significance arising from the Year 2000 challenge and 
its effects on government programs and service delivery, including the government’s progress in making 
its systems Year 2000 compliant. 


Audit Team 


Assistant Auditor General: Doug Timmins 
Principal: Nancy Cheng 
Director: Maria Wisniowski 


Brian Element 
Peter Taylor 


For information, please contact Nancy Cheng. 
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The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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Main Points 


21.1 In our 1993 Report, we raised a number of concerns about the performance of CIDA’s Bilateral Programs. 
Our 1995 and 1996 Reports contained a description of CIDA’s actions to implement an ambitious program of 
change and renewal. In this chapter we report on the extent to which CIDA has resolved the main concerns we 
raised in 1993. 


21.2 CIDA has adopted the concept of results-based management to change the Agency into a more 
results-oriented and learning organization. Progress is evident. CIDA has introduced a Results-Based Management 
Policy Statement, and supporting policies and guidelines have been developed and communicated to staff. It has 
established a Framework of Results and Key Success Factors as a guide for its managers and partners to focus 
more on results as they manage projects. However, progress in applying the principles of results-based 
management to CIDA contracts has been slow. 


213 CIDA is in the process of developing, reviewing and updating Regional/Country Development Policy 
Frameworks for all major countries and regions. Some of the more recently developed Frameworks state 
objectives and expected results. However, the expected results are not compared with actual results, one of the key 
means of assessing performance. 


21.4 At the project level, where official development assistance (ODA) is actually delivered, we found a 
greater emphasis on results, but uneven progress toward actually managing for results. Critical assumptions for 
project success and expected results have been articulated and a system of annual progress reporting implemented, 
but the statements of expected results are not always realistic. Further, once critical risks have been identified and 
assessed, they are not consistently monitored and addressed. This calls into question whether the expected benefits 
of projects will be realized. Projects are not systematically monitored after funding ends to determine whether the 
results expected have been achieved. 


21.5 CIDA prepares internal reports on performance, reflecting its increased ability to assess and report 
systematically on its performance. It also submits an annual Performance Report to Parliament; this would be 
improved if CIDA showed progress toward results for individual projects in the context of the results expected. In 
reviewing the Performance Report, we concluded that more rigour is needed to ensure that information is clear, 
meaningful, and accurate. The delivery of ODA in a great many countries and under uncertain political and 
economic conditions is a very complex and challenging undertaking, one that results in varying degrees of 
success. We believe it is important that CIDA bring out this dimension in its reporting; a more balanced picture 
would enhance the credibility of its reports. 


21.6 We concluded that CIDA’s actions have addressed the main concerns raised in 1993. The Agency now 
needs to keep up its momentum in implementing management for results. It has much of the supporting 
framework in place. To close the accountability loop, it needs to continue working on the measurement and 
reporting of development results. 
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Introduction 


21.7 CIDA is responsible for 
administering about 78 percent of 
Canada’s International Assistance 
Envelope (IAE). The remainder is 
delivered by, among others, the 
Department of Finance, the Department of 
Foreign Affairs and International Trade 
and the International Development 
Research Centre. 


21.8 CIDA delivers its international 
assistance through a variety of channels. 
Geographic programs, also referred to as 
bilateral or country-to-country programs, 
account for nearly half of its program 
expenditures. The rest is delivered through 
funding to multilateral institutions such as 
United Nations programs, to international 
financial institutions such as the Regional 
Development Banks, and to a multitude of 
non-governmental organizations including 
universities, businesses and not-for-profit 
organizations. 


21.9 As a result of Program Review, 
Canada’s budget for international 
assistance was cut from $2.6 billion in 
1994-95 to $2.1 billion in 1997-98; CIDA 
expenditures for Official Development 
Assistance (ODA) within this envelope 
were reduced from $2.2 billion in 
1994-95 to $1.7 billion in 1995—96 and 
have remained relatively constant since 
then (see Exhibit 21.1). In April 1995, the 
government transferred the management 
of the program for Central and Eastern 
Europe and the Newly Independent States 
from the Department of Foreign Affairs 
and International Trade to CIDA. 
Although the program does not fall within 
the ambit of Canada’s ODA, it is intended 
to be consistent with the foreign policy’s 
six priorities of official development 
assistance. 


21.10 For 1998-99, the IAE budget is 
about $2 billion; $1.6 billion of this will 
be spent on CIDA programs. Bilateral 
programs (Geographic Programs and 
Bilateral Food Aid) account for 
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approximately $790 million, or about 
50 percent of the Agency’s total 
expenditures (see Exhibit 21.2). 


Focus of the audit 


21.11 = This audit was the third and final 
phase of a planned follow-up. Phase I 
consisted of two parts: first, the Agency’s 
self-assessment of the actions taken as 
part of its renewal to address the principal 


This audit was the 
third and final phase of 
a planned follow-up of 


concerns raised in our 1993 Report; and our 1993 work. 
second, its development of a Performance 
Measurement Model (now called the 
Bilateral Project Performance Review 
System). Phase II covered CIDA’s 
progress in implementing the model to 
measure and report on the performance of 
its bilateral programs and projects. This 
audit was Phase III and followed through 
on our 1993 work. Details on how we 
approached this work are presented at the 
end of the chapter in About the Audit. 
21.12 Because ODA is delivered 
through projects, our audit was focussed 
primarily at the project level. We 
reviewed the Agency’s programming 
framework in the countries selected for 
audit and its linkage with projects in those 
($ millions) Exhibit 21.1 
Total International Assistance 
3,000 — Envelope — CIDA and Other 
Departments and Agencies, 
1994-95 to 1998-99 
2,500 — 
2,000 — Other Departments and [_] 
Agencies (Estimates) 
CIDA Central and Eastern [_} 
1,500 ~ Europe Program 
Expenditures (Actual) 
1,000 — CIDA Official fl 
Development Assistance 
Expenditures (Actual) 
500 — 
0 - 


1994-95 1995-96 1996-97 1997-98 1998-99* 


Source: Public Accounts; Estimates* 
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Exhibit 21.2 


Main Components of CIDA’s 
Share of the International 
Assistance Envelope, 1998-99 


($ millions) 


Bilateral programs account for 
approximately $790 million, 
or about 50% of the Agency’s 
total expenditures. 


Source: Part II] Estimates 


countries. We examined 28 projects in 
five countries, paying particular attention 
to planning, managing for, measuring and 
reporting results. In the course of our 
work, we heard many favourable 
comments by CIDA’s country partners 
about the dedication and co-operation 
shown by CIDA staff. Our 
country-specific findings are presented on 
pages 21-18 to 21-22. 


Observations and 
Recommendations 


Resolving Conflicts Among 
Objectives 


Aid is tied more closely to foreign policy 
objectives 


21.13 In 1993, we were concerned that 
it was difficult for CIDA to concentrate on 
its objective of putting poverty first and 
encouraging self-reliance while, at the 
same time, it had commercial and political 
objectives that did not always lend 
themselves to dealing with poverty ina 
direct way and that encouraged external 
dependency. 


21.14 We felt that competing or 
conflicting objectives should be pursued 
only consciously and in full realization of 


Geographic Programs 
($673) 


Bilateral Food Aid 
($115) 


the impact on the primary objectives. At 
stake was CIDA’s ability to pursue its 
development mandate effectively. 


21.15 In 1995, the government released 
its foreign policy statement Canada in the 
World, which set out a new mandate for 
Canada’s Official Development Assistance 
(ODA), and defined international 
assistance as an integral instrument of 
foreign policy: 


The purpose of Canada’s ODA is to 
support sustainable development in 
developing countries in order to 
reduce poverty and to contribute to a 
more secure, equitable and prosperous 
world. 


21.16 = Canada in the World mandates 
Canada’s ODA to concentrate available 
resources On Six program priorities: Basic 
Human Needs; Women in Development; 
Infrastructure Services; Human Rights, 
Democracy and Good Governance; 
Private Sector Development; and 
Environment. 


21.17 ~=Canada in the World helped 
CIDA to address problems of conflicting 
objectives. It led to a policy framework 
that provides a structure for linking 
CIDA’s development efforts to Canada’s 
foreign policy more rationally than had 
been the case under the previous strategy. 
Further, it has helped CIDA to 


International Financial 
Institutions ($149) 


Non-governmental 
Organizations ($190) 


Multilateral Food Aid and 
Technical Co-operation ($124) 


Industrial Co-operation 
and Other ($77) 


International Humanitarian 
Assistance ($54) 


Countries in Transition 
Program ($91) 


Administration ($110) 
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communicate and explain its work to its 
partners and stakeholders. 


21.18 CIDA budgets its bilateral ODA 
on a country basis. The need to 
concentrate assistance efforts on a limited 
number of countries has been a 
long-standing issue. In Canada in the 
World, the government committed itself to 
doing so while maintaining programs in 
other countries through low-cost, 
administratively simple delivery 
mechanisms. Over 120 countries received 
Official Development Assistance from 
Canada in 1996-97; the 20 largest 
beneficiaries accounted for just under 

63 percent of the total amount. This 
represents an increase from 1993-94, 
when the 20 largest beneficiaries received 
58 percent. As shown in Exhibit 21.3, the 
amount of assistance provided to countries 
rated by the Organization for Economic 
Co-operation and Development (OECD) 
as “least developed” and “other low 
income” has remained constant at about 
$458 million, or 55 percent of CIDA’s 
ODA expenditures. 


21.19 = Canada in the World calls for 

25 percent of ODA to be committed to the 
program priority of Basic Human Needs, 
which includes emergency humanitarian 
($ millions ) 

250— 
200— 


|e 0 


100— 


0 


Other low 
income 
countries 


Least 
developed 
countries 
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assistance. In 1996-97, 38.4 percent of 
CIDA’s ODA program expenditures went 


to Basic Human Needs, including food aid 


and humanitarian assistance (33.9 percent 
when emergency humanitarian aid is 
excluded). 


Implementing Results-Based 
Management 


Progress toward results-based 
management is evident 


21.20 =As aresult of its 1992 Strategic 
Management Review and our 1993 report 
on its bilateral programs, CIDA launched 
an ambitious renewal initiative in 1994. It 
made a commitment to Parliament to 
transform the Agency into a more 
results-oriented, accountable organization. 
CIDA adopted a policy of results-based 
management as an Agency-wide initiative 
to enable it to systematically address these 
commitments (see Exhibit 21.4). 


21.21. To make the policy operational 
and to integrate the concept of 
results-based management into project 
delivery, the Agency undertook several 
key steps: 


e developed an Agency Accountability 
Framework; 


| 


Lower-middle Upper-middle Regional 
income income programs/ 
countries countries not classified 


In 1994 CIDA made a 
commitment to 
Parliament to 
transform the Agency 
into a more 
results-oriented, 
accountable 
organization. 


Exhibit 21.3 


CIDA Official Development 
Assistance — Comparison of 
Disbursements by OECD Rating, 
1993-94 and 1996-97 


[_] 1993-94 
1996-97 


Source: International 
Development Information 
Centre: Statistical Report on 
Official Development 
Assistance 
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CIDA’s accountability 
framework requires it 
to identify its 
objectives and 
demonstrate that 
resources are 
managed to achieve 
the intended results. 


Exhibit 21.4 


CIDA’s Results-Based 
Management Policy Statement 


e assembled practices, policies and 
guidelines into a Roadmap for 
practitioners; 


e articulated its Framework of Results 
and Key Success Factors; 


e developed Regional/Country 
Development Policy Frameworks; and 


e provided extensive training. 


These and other actions represent 
significant advances by the Agency in 
putting into place the supporting 
framework for implementing results-based 
management. 


An accountability framework has been 
developed 


21.22 In 1993 we highlighted a need to 
clarify the extent to which CIDA is 
accountable to Parliament for managing 
for results, and the extent to which CIDA’s 
staff and its partners are respectively 
accountable for obtaining results. In July 
1998, CIDA’s executive committee 
approved an accountability framework. It 
identifies accountabilities of the Agency 
overall; it further articulates the 
accountabilities of the President, branch 
heads, and key managers in each branch. 


21.23. ~The basic definition of 
accountability in the framework requires 
that CIDA identify its objectives and 
demonstrate that the resources allocated to 


it for official development purposes are 
managed to achieve the intended results. It 
also requires that the Agency report the 
results achieved in its development 
programs to Parliament and the Canadian 
public. The framework indicates that, 
through its partnerships with developing 
countries and Canadian and international 
partners, CIDA shares accountability for 
development results and its own 
accountability must be viewed in that 
light. CIDA accepts responsibility and 
accountability for monitoring actions by 
its recipient partners as well as other 
events that may affect development goals, 
and for acting to ensure that momentum 
toward these goals is maintained. 


21.24 This shared accountability for 
ODA delivery makes it especially 
important for CIDA to follow through on 
its programs and projects so it can be ina 
position to report on whether Canada’s 
ODA is producing the expected results. 
This type of information would be useful 
to include in CIDA’s Performance Report 
to Parliament. 


21.25 CIDA accepts that it is fully 
accountable for achieving operational 
results. This means that it is fully 
accountable for the setting of objectives, 
formulation of policies, selection of 
development initiatives, allocation of 
resources, monitoring and performance 
reporting. This also involves identifying 
expected outputs and outcomes, assessing 


Results-based management in CIDA means: 


e defining realistic expected results, based on appropriate analyses; 


e clearly identifying program beneficiaries and designing programs to meet their needs; 


* monitoring progress toward results, and resources consumed, with the use of appropriate indicators; 


e identifying and managing risks, while bearing in mind expected results and the necessary resources; 


e increasing knowledge by lessons learned, and integrating them into decisions; and 


* reporting on results achieved and the resources involved. 


The Policy defines the results chain for bilateral program and project as follows: 


© output is the immediate, visible, concrete and tangible consequence of program/project inputs; 


e outcome is the result at the purpose level, constituting the short-term effect of the program/project; 


e impact is a broader, higher-level, long-term effect or consequence linked to the goal or vision. 
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related risks and monitoring them, and 
taking appropriate corrective action. 


Work on results-based contracting has 
slowed considerably 


21.26 The Agency has not been as 
successful in its attempt to develop 
results-based contracting. CIDA delivers 
nearly all bilateral ODA through contracts 
with Canadian executing agents (CEAs). 
In 1995, it recognized the need for a 
contracting approach that would define 
the respective accountabilities and risks of 
both the executing agents and CIDA. In 
1996, CIDA consulted with its CEA 
community. A joint task force, 
representing CIDA and the community, 
defined results-based contracting as “a 
contracting method which employs an 
iterative and participatory approach to 
results definition and seeks to promote the 
achievement of development results 
through appropriate contractual terms 
such as the statement of work, the basis of 
payment (including incentives) and 
mutually accepted performance 
indicators.” 


21.27‘ The report by the task force in 
June 1996 expressed some concerns about 
this approach; to address them it 
articulated a number of guiding principles. 
It called on the Agency and the supplier 
community to use the ideas and 
suggestions in the report and further 
develop the necessary policies, programs 
and operational practices needed to 
implement results-based contracting. 
However, CIDA deemed them too 
impractical to implement. Since then, 
progress has slowed considerably. The 
Agency told us that the complexity of 
trying to develop a results-based 
contracting approach was much greater 
than expected. In July 1998, a committee 
chaired by the President was established 
to review and simplify CIDA’s contracting 
regime. Part of its work will be to 
recommend changes needed to implement 
an approach to contracting for results. 


_ improvements in the way CIDA manages 
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21.28 CIDA should make every effort 
to identify practical steps that it can 
take to incorporate the principles of 
results-based management in its 
contracts, and experiment with these 
steps in selected projects. 


CIDA’s response: CIDA will continue to 
follow the course of action developed by 
the Committee for Improving Contracting. 
Based on the results of pilot projects, it 
will pursue a practical and iterative 
approach to contracting for results and 
apply the lessons learned, where feasible, 
to its contracting process, recognizing that 
this is a long-term endeavour. 


Risk management and monitoring need 
to be more consistently integrated into 
projects 


21.29 Our 1993 audit of CIDA’s 
bilateral programs noted the need for 


In July 1998, a 
committee chaired by 
the President was 
established to review 
and simplify CIDA’s 
contracting regime. 


its operating risks, and CIDA then took 
steps to improve its risk management. 
Project managers were expected to 
monitor anticipated risks and use the 
information thus gathered to adjust project 
activities as necessary. CIDA’s 
Results-Based Management Policy 
Statement and its Framework of Results 
and Key Success Factors called risk 
management integral to results-based 
management at the level of both country 
and project. We found that CIDA has 
improved its guidance and training on the 
identification and assessment of risk. 
Managers are trained to include in 
approval documents critical assumptions 
about risks to the outputs, outcomes and 
ultimate impacts of projects. However, 
there is little practical guidance on how to 
collect and use information on project 
risks, except for financial risk, as these 
evolve over time. CIDA would benefit 
from developing more specific guidance 
and tools in this area. 


21.30 Our 1993 audit underlined the 
need to be able to obtain early warnings of 
project difficulties. In most of the projects 
we examined, we found limited ongoing 
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The sustainability of 
benefits needs to be 
considered throughout 
the duration of CIDA’s 
involvement and 
monitored afterward. 


Exhibit 21.5 


Four Key Indicators of Project 
Self-Sustainability 


monitoring of risks that had been 
identified in the planning phase, and no 
deliberate strategy for managing them. In 
their project reports to CIDA 
headquarters, Canadian executing agents 
normally discussed activities, not risks. In 
a number of projects, problems arose that 
had been at least partially predicted 
through risk analysis in the planning 
phase. 


21.31. CIDA guidance and training have 
so far dealt with external risks — those 
outside the direct control of project 
managers, and often related to the level of 
commitment by the recipient country. The 
guidance does not address the equally 
important job of managing risks that are 
more internal to the project — for 
instance, those related to critical tasks, 

key personnel, or project complexity. 


21.32 CIDA should broaden its 
guidance on risk assessment and risk 
management to include internal risks as 
well as external risks to a project. 


CIDA’s response: CIDA will strengthen its 
identification, assessment and 
management of internal risks of projects 
by enhancing the guidelines in the 
bilateral Roadmap and by reinforcing risk 
management in its training on 
results-based management. 


Determining whether benefits have 
endured is essential 


21.33 In order for development to 
occur, the benefits from a project should 
last beyond the termination of aid funding. 
The sustainability of benefits needs to be 
considered throughout the duration of 


CIDA’s involvement, and monitored 
afterward to determine whether expected 
benefits have endured. Exhibit 21.5 lists 
four key indicators of project 
self-sustainability. We noted that when 
events put at risk these conditions for 
sustainability, reactions by project 
managers were inconsistent. Some took 
action to deal directly with the risk. 
Others tended to continue the project with 
minor adjustments or consider applying 
additional resources, rather than stand 
back and analyze more fundamentally 
whether the project should be substantially 
changed or discontinued. 


21.34 Generally, sustainable 
programming requires commitment to 
projects over a long time frame — 
possibly 10 to 15 years. The commitment 
must be supported by a strategy 
addressing the level of programming that 
is appropriate, the participation of other 
donors, and the recipient country’s 
capacity to assume the project activities 
fully once CIDA’s funding ends. We found 
that CIDA’s five-year planning horizon 
does not cover the period of time that can 
be expected to elapse before the outcomes 
of these longer-term projects begin to take 
effect. Consequently, some of the projects 
cannot be sustainable unless their initial 
phases, usually five years, are extended by 
another one or two phases; yet planning 
documents do not always reflect this 
consideration. 


21.35 CIDA and other donors are 
leaning increasingly toward “softer” 
projects, such as the institution- 
strengthening and capacity-building 
projects we examined. Because managing 
and measuring results achieved from these 


Technical and managerial capability or readiness of the recipients (recipient country or 


“inheritor” institution) to sustain the project. 


Commitment of the “‘inheritor” to assume ownership and responsibility for sustaining the project. 


Target group involvement in project design and evaluation, and the value it places on the 


development benefits. 


Availability of sustaining resources — cash and other resources — to operate and maintain the 


project over its life cycle. 


Report of the Auditor General of Canada — December 1998 


projects present a new set of challenges, 
and the projects are related indirectly to 
poverty reduction, CIDA needs to closely 
monitor its own experience and that of 
others to ensure that this type of program 
tool is producing the expected results. 


21.36 Applying results-based 
management to this softer type of project 
is particularly challenging. For example, 
expected results of development are hard 
to define and perhaps even harder to 
measure in projects that involve institution 
strengthening and capacity building. In 
most cases, longer-term outcomes and 
impacts of development do not begin to 
show until years after CIDA’s direct 
involvement in funding ends. 


21.37. ~=To focus on results, however, 
CIDA needs to monitor whether the 
expected shorter-term effects of a project 
have been achieved and are likely to lead 
to the desired longer-term development 
impacts. Even though a project may be 
finished and CIDA funding ended, it is 
still important to monitor in a 
cost-effective way whether the expected 
results have materialized. We found that 
CIDA does not systematically continue to 
monitor projects after their completion to 
determine whether the results expected in 
the short and long terms have been 
achieved. Lack of this information 
precludes both possible action to increase 
the prospects of long-term results and an 
opportunity to learn lessons for future 
projects. Currently, CIDA obtains this 
kind of information in a limited way only 
through periodic performance reviews of 
its development priorities. 


Reporting on Project Performance 


The quality of performance reporting is 
inconsistent 


21.38 Although the elements of 
results-based management are becoming 
embedded in project delivery, the first key 
step — defining results that are realistic 
— has not been easy. In most of the 
projects we examined, the initial 
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statements of expected results — 
particularly long-term impacts — were 
unrealistic, resulting in a subsequent need 
to modify the project or its goals, and 
presenting problems for reporting against 
expected results. 


21.39  Results-based management 
guidelines and policies issued in 1996 
refer frequently to expected results, and to 
comparing them with actual results. In 
practice, however, the focus has been on 
the immediate or short-term results of 
projects. Although statements of expected 
impacts or longer-term effects are 
developed along with quantitative or 
qualitative indicators of them, they are not 
integrated into the project management 
system for aid delivery. For example, 
there 1s no provision for longer-term 
monitoring to assess these indicators and 
report on whether development goals have 
been achieved. 


21.40 = The Policy for Performance 
Review, the Results-Based Management 
Policy Statement, and the accountability 
framework provide the structure for 
CIDA’s performance measurement and 
reporting systems. The Annual Project 
Progress Report system (APPR), begun in 
1996, provides the basic tool for tracking 
active projects with a value over 
$100,000. 


21.41 Once project managers complete 
these progress reports, branch 
management reviews them, and uses them 
to track the results of the projects. They 
are also used at the country and branch 
levels to address the difficulties identified 
in them. The information in the APPR 
includes project expenditures, expected 
and actual results (outputs, outcomes and 
impacts), lessons learned and progress 
ratings. The individual reports are 
compiled and a Bilateral Branch 
Achievement Report is produced. These 
reports along with audit and evaluation 
reports are used as inputs to prepare the 
Agency’s Performance Report to 
Parliament. 


The Annual Project 
Progress Report 
system is still 
evolving. 
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The Performance 
Report illustrates 
achievements, but 
does not describe 
them in the context of 
the results that were 
expected. 


21.42 We reviewed the 1997-98 APPRs 
on the projects we had selected for our 
audit to see whether, based on our 
understanding of the projects, the 
information provided was accurate and 
meaningful. We found that most of the 
reports reflected the status of the projects 
fairly. However, in some APPRs we noted 
that the information was stated in terms 
that were more positive than the situation 
warranted. Also, actual results could not 
always be readily compared with the 
statements of intended immediate and 
short-term results. The Annual Project 
Progress Report system is still evolving. 
More management attention is needed, in 
our view, to ensure that the information 
provided in the reports is accurate and 
meaningful and that it is used in deciding 
on the future direction of projects. 


CIDA’s Performance Report to 
Parliament 


More meaningful, accurate and 
balanced reporting is needed 


21.43 Along with other government 
departments, CIDA submitted its 1996-97 
Performance Report to Parliament in the 
fall of 1997. To determine how its 
performance reporting system works in 
practice, we reviewed the way CIDA 
produced the Geographic Programs 
section of its Performance Report for the 
fiscal year ended 31 March 1997. The 
Annual Project Progress Reports (APPRs) 
are the main source of the project 
information presented in the Performance 
Report; the latter cites many examples of 
the results produced by CIDA projects. 
We compared the results of 33 
illustrations shown in the section on 
Geographic Programs with the results 
shown in the APPRs for the same projects. 


21.44 The Performance Report 
describes a number of projects to illustrate 
performance. The Agency points out that 
international development activity does 
not yield meaningful results in neat 
financial year intervals. Consequently, 


most of the illustrations present the results 
of activities that span a number of years, 
but the distinction is rarely made between 
these results and others that were the 
result of only one year’s activity. Where 
the results shown cover several years of 
activity, the number of years is not shown. 
No information is given on the resources 
used to achieve the results shown for 
specific projects. These resources may 
include not only CIDA’s expenditures but 
also the human and financial resources 
provided by other donors and recipient 
governments. Also, while the illustrations 
describe achievements, they do not 
describe them in the context of the results 
that were expected. Nor do they describe 
the significance of these specific projects 
in the context of the overall development 
program for the country. This information 
would provide a more meaningful picture 
of the Geographic Program’s performance. 


21.45 Wealso noted instances where 
the stated results were not consistent with 
those in the Annual Project Performance 
Reports (APPRs). These included, for 
example, showing expected rather than 
actual results or results beyond those 
shown in the performance reports. In our 
view, more rigour is needed in reviewing 
and compiling information for the 
Performance Report to ensure that it is 
clear and accurate. 


21.46 CIDA delivers ODA in many 
countries under uncertain political and 
economic conditions and enormous 
constraints of infrastructure and 
geography. This makes project success 
and assessment of development results 
very complex and challenging. We noted 
that the branch achievement reports 
include numbers and percentages of 
projects that are progressing satisfactorily, 
those that have manageable problems and 
those with serious problems. However, the 
1996-97 Performance Report excludes 
this information, although the Agency has 
reported it in the past. We believe it is 
important that CIDA bring out this 
dimension in its reporting; the credibility 
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of its reports would be enhanced by a 
more accurate balance between positive 
accomplishments and areas where 
expected results could not be achieved. In 
the latter cases, CIDA could describe the 
types of actions it has taken to deal with 
problems or the way it will apply lessons 
learned to future projects. 


21.47. CIDA should more rigorously 
review the quality of the information in 
its Performance Report, and present a 
more meaningful and balanced picture 
of its performance. 


CIDA’s response: CIDA will continue to 
improve the consistency, accuracy and 
reliability of its performance information 
so that a more balanced and meaningful 
picture can be reflected in its Performance 
Reports. 


Core indicators to measure development 
progress are being developed 


21.48 In its Performance Report, CIDA 
acknowledged that it is possible to 
measure outputs (the immediate, visible, 
concrete and tangible results of a project) 
and outcomes (the achievement of the 
purpose identified for the particular 
project). However, it questioned the 
feasibility of using aggregated 
performance information from the project 
and institutional levels to comment 
definitively on the Agency’s overall 
performance or on its performance at the 
country or program level. 


21.49 The Agency points to the 
difficulty of measuring impacts — that is, 
broader, higher-level, long-term benefits 
to the community, country or group. There 
are several factors that make the task 
highly problematic. A key one is the 
difficulty of attributing a particular 
development impact to a specific CIDA 
contribution. A second is delay: generally, 
outcomes and impacts can be expected 
only several years after projects have been 
completed. It is difficult to track them 
after the beneficiaries have taken 
ownership of a project. CIDA points out 
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that other donors also experience these 
difficulties. The Agency is sharing with 
them its approaches and its performance 
data to find ways of addressing these 
concerns. 


21.50 We agree that it may be difficult 
to measure outcomes and impacts and to 
attribute them to a single project or a 
single donor such as CIDA. Nonetheless, 
the Agency allocates bilateral program 
funds on a country or regional basis. The 
strategic planning process culminates in a 
multi-year plan at the country or regional 
level, based on an analysis of the various 
factors and risks underlying CIDA’s 
interventions. We noted that some of the 
more recent Regional/Country 
Development Program Frameworks state 
the development results that are expected 
at the country level. CIDA could attempt 
to periodically assess whether these 
expected results have been achieved at the 
regional/country level. 


21.51 Reporting on Canada’s ODA 
program is often done in terms of amount 
spent — usually as a percentage of Gross 
National Product (GNP), which is then 
compared with the standard international 
spending target for ODA, 0.7 percent of 
GNP. In 1996-97, Canada’s spending on 
ODA was about 0.34 percent of GNP. This 
indicator is widely used internationally for 
intercountry comparisons of spending on 
official development assistance. 


21.52 While this may be a useful 
indicator of spending on aid effort or 
commitment to it, it is an indicator of 
input rather than output: it does not 
measure the results or the effectiveness of 
aid. Developing indicators of results is a 
logical next step in CIDA’s results-based 
management process. CIDA has been 
supporting and working with the 
Development Assistance Committee of 
the OECD to develop a working set of 
core indicators for measuring the progress 
of development efforts. An illustration of 
these possible indicators is shown in 
Exhibit 21.6. Developing and reaching 
international consensus on core indicators 


Reporting on Canada’s 
Official Development 
Assistance program as 
a percentage of GNP 
does not measure the 
results or 
effectiveness of aid. 
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Exhibit 21.6 


Measuring Development Progress: A Working Set of Core Indicators 


Goals 


Indicators 


Economic Well-Being 


Reducing extreme 


The proportion of people living in extreme 


Incidence of extreme poverty: population below 


of development 


Gross National Product 


Adult Literacy Rate 
Total Fertility Rate 
Life Expectancy at Birth 


poverty | poverty in developing countries should be $1 per day. 
reduced by at least one-half by 2015. 2. Poverty gap ratio: incidence times depth of 
poverty 
3. Inequality: poorest fifth’s share of national 
consumption 
4. Child malnutrition: prevalence of underweight 
under 5s 
Social Development 
Universal primary | There should be universal primary education in 5. Net enrolment in primary education 
education | all countries by 2015. 6. Completion of 4" grade of primary education 
7. Literacy rate of 15 to 24 year-olds 
Gender equality | Progress towards gender equality and the 8. Ratio of girls to boys in primary and secondary 
empowerment of women should be education 
demonstrated by eliminating gender disparity 9. Ratio of literate females to males 
in primary and secondary education by 2005. (15 to 24 year-olds) 
Infant and child | The death rates for infants and children under 10. Infant mortality rate 
mortality | the age of five years should be reduced in each 11. Under 5 mortality rate 
developing country by two-thirds the 1990 
level by 2015. 
Maternal mortality | The rate of maternal mortality should be 12. Maternal mortality ratio 
reduced by three-fourths between 1990 and 13. Births attended by skilled health personnel 
AAQISY, 
Reproductive health | Access should be available through the 14. Contraceptive prevalence rate 
primary health-care system to reproductive 15. HIV prevalence in 15 to 24 year-old 
health services for all individuals of pregnant women 
appropriate ages, no later than the year 2015. 
Environmental Sustainability and Regeneration 
Environment | There should be a current national strategy for 16. Countries with national sustainable development 
sustainable development, in the process of strategies 
implementation, in every country by 2005, so 17. Population with access to safe water 
as to ensure that current trends in the loss of 18. Intensity of freshwater use 
environmental resources are effectively op Ae) 
; 19. Biodiversity: land area protected 
reversed at both global and national levels by ek ‘ : : 
2015. 20. Energy efficiency: GDP per unit of energy use 
21. Carbon dioxide emissions 
General Indicators 
Other selected indicators | Population GNP per Capita Aid as % of GNP 


External Debt as % of GNP 
Investment as % of GNP 
Trade as % of GNP 


Source: OECD 
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of results is a complex undertaking, and 
the current effort is still very much a work 
in progress. We encourage CIDA to keep 
up its efforts with other donors, 
multilateral institutions and developing 
countries to refine and reach consensus on 
a set of indicators. 


21.53 Once this is done, CIDA could 
use the indicators in its reports to describe 
a country’s development results. Because 
donors are acting more in concert, the 
results will likely not be attributable 
directly to any one donor, but CIDA could 
show how its projects have contributed to 
the overall results. What is important is 
that lasting development results be 
achieved, not that they be attributed 
directly to the intervention of any 
particular donor. This type of reporting 
would reinforce CIDA’s move to focus 
more on development results, and would 
improve accountability for the effective 
use of ODA funds. 


Counterpart Funds 


CIDA has responded to 1993 concerns 
but better control is needed over 
payments into counterpart funds 


21.54 In June 1994, CIDA published a 
set of new directives on counterpart funds. 
The new directives defined the respective 
accountabilities and responsibilities of 
CIDA and the recipient country for 
managing counterpart funds. Counterpart 
funds are a development instrument 
whereby Canadian commodities or goods 
are converted to aid funding. CIDA 
approves a project amount, which it refers 
to internally as a line of credit to the 
recipient country, to finance the purchase 
of the commodities or goods in 
accordance with its Canadian content 
policy. A Canadian supplier sells the 
product in the recipient country and CIDA 
pays the supplier in hard currency. The 
policy requires that normally the same 
amount in local currency then be paid into 
the counterpart fund by the local buyer as 
described in the agreement with the 
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recipient country. The counterpart fund 
legally belongs to the recipient country; it 
is usually controlled by a joint board and 
used to fund development activities. In 
1993, we were concerned about CIDA’s 
lack of assurance that all moneys in 
counterpart funds had been spent for the 
intended purposes and had been accounted 
for fully. 


21.55 In 1997, CIDA undertook an 
audit of counterpart funds. An interim 
audit report in March 1998, based on the 
audit of three funds, noted weaknesses in 
the flow of money into all three that had 
resulted in the appropriate amounts not 
being credited to the funds. In one case, 
the internal audit could not provide 
assurance that the recipient government 
had deposited into the fund the full 
amount that was required, and that funds 
had been used for the purposes intended. 


The audit also concluded that CIDA 


should re-examine the issue of the 
ownership of the funds as well as 
reporting on results achieved from their 
use. As a result, the counterpart fund 
policy is being revised to address these 
concerns. 


21.56 We examined three counterpart 
funds in Bolivia, Bangladesh, and 
Senegal. Given that the structure of each 
counterpart fund is unique to the country 
in which it operates, we cannot form an 
opinion on the management of counterpart 
funds in general. We found that for the 
funds we examined, CIDA had adequate 
assurance that all moneys had been spent 
for development purposes and had been 
accounted for fully. However, in two of 
the funds we found problems in control 
over the flow of moneys that resulted in 
less than the full amounts being placed in 
the funds. In one case, CIDA allowed 
payment into the fund of less than it had 
paid to the Canadian supplier (see Exhibit 
21.7). In the other fund, the memorandum 
of understanding with the recipient 
country allowed it to withhold from the 
amount to be deposited up to 7.5 percent 
in incidental charges. We noted that there 


(continued on page 21-22) 
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Bangladesh 


Bangladesh is one of the Agency's largest 
bilateral programs. In 1997-98, CIDA’s 
expenditures on the program were $37 million; 
over the past five years they have totalled about 
$225 million. 


In 1991, CIDA developed a Country 
Development Policy Framework for 
Bangladesh. The framework pointed to many 
structural weaknesses in the country, such as 
weak state institutions and a lack of local 
financial resources to operate and maintain 
development projects. These made self-reliant 
development very difficult. The goals and 
objectives of Canada’s ODA program were 
long-term. They aimed at supporting the reform 
process in Bangladesh by influencing policy 
changes and strengthening institutional capacity 
for sustainable development, and by 
transforming donor dependency into a 
partnership based on greater self-reliance. This 
represented a move away from CIDA’s former 
capital-intensive approach in the energy and 
railway sectors. 


CIDA is currently finalizing a new program 
framework for Bangladesh. The new framework 
will not represent a dramatic departure from the 
direction outlined 1991, given that the 
fundamentals underlying Bangladesh’s 
development dilemma have not changed 
significantly since then. 


Bangladesh was one of the countries 
selected for our 1993 audit. Sustainability of 
projects was one of our major concerns. Some 
of the projects included in that audit are still 
ongoing. During our 1998 audit we wanted to 
see what changes the Agency had made to 
improve sustainability. We selected five 
projects; three had been in existence since 
before 1993, and two of those were among 
projects we examined in 1993. Since then, 
CIDA has made major changes in the projects 
to enhance their potential for sustainability. In 
general, we found that CIDA had taken steps to 
address our 1993 concerns. 


To enhance the possibility of sustainable 
developmental results, one project we 
examined was redesigned as a result of lessons 
CIDA had learned during the initial years of 
implementation. 


The Rural Maintenance Project was 
initiated in 1983 as a small pilot project. It is 
now in its third phase. Phase | (1985-89) was 
originally designed as a relief project to employ 
destitute women to maintain rural roads 
year-round. There was no limitation set on the 
duration of employment during the early period 
of the project, and there were no plans for the 
transfer of the project to the Government of 
Bangladesh. Phase II saw a further expansion 
of the project. CIDA undertook evaluations 
during the course of the project in 1992 and 
1993. As a result of the 1992 evaluation, the 
project was redesigned. Phase II saw the 
inclusion of an income diversification 
component. 


A “graduate” of the 
Rural Maintenance 
Project now selling 
clothing to earn a living. 


The income diversification component of 
the redesigned project provides skills training to 
the women so that when they leave the project 
they can earn a living. During their four years of 
employment, the women accumulate savings 
through a 19 percent holdback in wages, and 
are trained in ‘life skills” such as arithmetic and 
household budgeting and in possible means of 
self-employment for income generation. They 
“graduate” from the program after four years, 
and others are recruited into it. During Phase 
Ill, which started in 1995, the redesigned 
project employed approximately 36,000 women. 


The project provides for the gradual 
transfer of more ownership and responsibility 
for road maintenance from CIDA and the 
Canadian executing agent to a ministry of the 


Government of Bangladesh. Financial 
responsibility for the payment of salaries to the 
women is increasingly borne by the 
Government of Bangladesh. The skills 
development and training components of the 
project are also being shifted to a new 
Bangladeshi entity. 


The 1995 agreement with the Government 
of Bangladesh stipulated that it would increase 
its contribution to the women’s salaries to 
90 percent from the original 10 percent by the 
end of Phase III in June 2001. In 1998 CIDA 
agreed to reduce this commitment to 55 percent 
by July 2000. In 1998, the Bangladesh 
contribution was 35 percent. 


Another project we examined showed a 
delay in effective action to ensure stronger 
commitment by the recipient organization. 
Warning signs that called the project's 
sustainability into question had been present for 
several years. 


The Bangladesh Rail |I project was 
approved in 1986 at a value of $99 million over 
a period of 10 years. It was aimed at improving 
the capabilities and performance of repair and 
overhaul workshops. Stage I! was started in 
1994. In 1995, the agreement with the 
Bangladesh government was extended to 
March 2000. The project goal was to increase 
locomotive availability, productivity and traffic 
volumes and thus Bangladesh Rail’s revenue. It 
was part of a larger strategy supported by 
another major donor that aimed to 
commercialize the railway, currently run by the 
government. 


Workers of the Rail II project 
overhauling locomotive components 
at the central locomotive workshop. 
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In 1990, after a major assessment of the 
viability and relevance of the project, CIDA set 
out the key conditions and the inputs required 
from Bangladesh Rail (BR) as a condition of 
CIDA’s continued involvement. Due to the size 
of the project and its complexity, CIDA engaged 
an agency to assist in the management of the 
project. Monitoring missions reported that 
although progress was being made, project 
results for Stage II were slipping. Bangladesh 
Rail was not living up to its promised level of 
support for the project. For instance, it did not 
provide the full budget to meet the target for 
locomotive overhauls. CIDA held meetings with 
BR and agreed to new targets. Again, these 
were not met, or were met only partially. It 
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became evident that from the outset the project 
had not enjoyed the necessary visibility and 
commitment among the top management of the 
railway. 


Finally, in June 1997, the project manager 
concluded that due to a lack of effective 
progress by Bangladesh Rail, the sustainability 
of the Rail lI project was in serious doubt. While 
continuing its funding, CIDA re-evaluated the 
project in the ensuing months and considered 
the options of continuing or not. As a result of 
the assessment, CIDA informed BR that 
continuation of the project was in doubt. BR 
senior management then pledged stronger 
commitment to remedy the situation and to 
support the project. It submitted a detailed 


time-bound action plan to address the concerns. 
It increased the maintenance budget and 
developed a plan for a new organizational 
structure for locomotive maintenance and rolling 
stock. 


CIDA decided to continue, but modified the 
scope of the project. It refocussed some 
aspects to link to the corporate reforms of BR 
that it considers essential to long-term 
sustainability. It plans to reassess the project on 
an annual basis to ensure that established 
targets are being met and that the 
achievements are indeed sustainable. CIDA’s 
disbursements to the end of 1997 totalled more 
than $70 million. 


Bolivia 


In 1995 CIDA designed a Bilateral 
Program Framework for Bolivia that set out 
the goals and objectives of Canada’s ODA 
program for the ensuing five years. The 
document identified program themes, a series 
of proposed program elements or projects, an 
assessment of risks, and a number of 
expected results. 


CIDA’s bilateral assistance to Bolivia is 
centered on two themes: 


e improving the long-term income-earning 
potential of poorer Bolivians; and 


e supporting the reform initiatives announced 
in a 1993 master plan by the Bolivian 
government and translated in 1994 into five 
priority reform programs. 


CIDA’s program themes were closely 
related to the reforms. The program it 
proposed would build on past Canadian 
assistance to Bolivia, notably in such fields as 
the hydrocarbon sector, the health sector, 
support to groups providing credit to 
micro-enterprise, and public sector reforms. 


At the 1994 meeting of the World Bank 
Consultative Group, the Government of 
Bolivia received strong indications of donor 
support. CIDA is one of the smaller donors to 


Bolivia, with a bilateral country program of 
$7 million per year in 1997-98. 


. i 


CIDA staff at the gas 
refinery near Santa 
Cruz, part of the state 
oil and gas company 
that was capitalized. 


CIDA identified the challenges and risks 
to achieving the expected results, and 
identified a number of principles to be 
followed to minimize those risks. A midterm 
review to assess progress against the 
expected results of the strategy was planned 
for 1998. 


We examined five projects In Bolivia. 
Many of CIDA’s projects in Bolivia were 
“softer’ in nature and were focussed on 
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capacity building. The projects appeared to 
demonstrate a strong focus on results. 


CIDA's largest project, at just over 
$11 million, involved the hydrocarbon sector. 
This project demonstrated a focus on results, 
good risk management and attention to 
sustainability. Project objectives and outputs 
were developed jointly by the recipient 
organizations and the executing agent. The 
recipient organizations nominated a 
“champion” at a senior level with 
decision-making authority. The Bolivian 
government provided access to its highest 
levels of decision making. During the project's 
implementation, regular technical and 
management review meetings took place with 
senior levels of government and the state oil 
and gas company. The recipient Bolivian 
organizations provided dedicated personnel 
and subsidized the costs of Canadian 
advisors. Monitoring the risk factors and 
adjusting the project accordingly contributed 
to moving the project forward. 


To date, large sections of the company 
have been capitalized, the legislation has 
been enacted, the regulatory body and the 
regulatory framework have been established 
and they are being administered by the 
Bolivian government. Those were among the 
expected results of the project. 
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Estonia 


CIDA manages its aid to Estonia under a 
transition strategy whereby aid will gradually 
decrease, and end definitively in 2005-06. The 
strategy was based on CIDA’s Regional 
Framework for Central and Eastern Europe. 
The Estonian government aims to expeditiously 
meet the necessary standards for participation 
in the European Union market. Our discussions 
with Estonian government officials and project 
partners demonstrated a general agreement 
with the CIDA strategy and deadline. 


Aid to Estonia is not classified as ODA. 
Project planning and implementation there have 
differed in some respects from ODA projects. 
Initiatives for projects often emerged 
domestically from Canadians with ties to 
Estonia, and CIDA’s involvement was usually 
responsive rather than proactive. CIDA has 
encouraged projects to acquire funding from 


The old city of Tallinn. CIDA has 
helped the City of Tallinn modernize 
its city planning function and create 
a tourism plan. 


other donors and has placed a strong emphasis 
on local involvement. The program has 
acknowledged the fast pace of change in 
Central and Eastern Europe and has 
encouraged flexibility in project planning and 


implementation. The program recognizes that it 
should not have unrealistic expectations for its 
overall impact in the countries. Its aim has been 
to support specific aspects of the reform 
process by addressing bottlenecks and 
capitalizing on opportunities within the 
transitional time frame. 


We audited 11 projects in Estonia. They 
included some projects that were also operating 
in Latvia and Lithuania. Most projects had 
resulted in clear sustainable benefits. We noted 
a consistent focus on results by CIDA 
managers, Canadian executing agents and 
local partners. We also noted some 
weaknesses in risk management. In the 
absence of ongoing risk management, CIDA 
dealt with the problems related to previously 
identified risks by applying additional resources 
through expansion or extension of the projects. 


Senegal 


In June 1997, CIDA established a country 
profile for Senegal. It describes the social, 
economic and environmental situation in 
Senegal, the local government strategy for 
development and the Canadian strategy for aid 
intervention. CIDA’s development strategy in 
Senegal takes into consideration the strategy of 
the Senegalese government and is in line with 
the Agency's strategy in West Africa. However, 
the country profile does not identify expected 
results, nor means to measure performance at 
the country program level. We also found that 
for most projects we examined in Senegal, the 
Senegalese contribution was provided mainly by 
the counterpart fund, financed entirely by CIDA. 
This new country strategy did not address that 
issue, although it is critical to ensuring the 
sustainability of results. At the time of our audit, 
CIDA told us that it was negotiating with the 
Senegalese government to address this 
situation. For the five-year planning period, 
CIDA’s programming in Senegal will focus on 
education, good governance and support to the 
private sector. Although CIDA is not one of the 
major donors, its support to Senegal has been 


consistent for many years at approximately 
$20 million per year. 


CIDA’s objective under education is to 
increase the child schooling rate and to 
decrease illiteracy. In Senegal, the level of 
schooling is among the lowest in the world and 
is still declining. Reversing that situation is an 
important objective for the government of 
Senegal and for CIDA. Three of the six projects 
we examined during our field audit in Senegal 
are related to education. Two mutually 
supportive projects to increase elementary 
schooling rates have so far shown a good focus 
on results. In 1994, CIDA started a development 
project with the objective of improving basic 
education facilities and access. The project had 
been planned in accordance with the 
Senegalese national plan for education and in 
collaboration with other international donors. It 
used an innovative approach to attract children 
to primary education, by bringing in the 
community to participate in the school program. 
Commitment by local communities and the 
transfer of expertise to beneficiaries were 
viewed as requirements for sustainability. CIDA 


managers adopted a results-based 
management approach to the project. 


In 1996, the other international partners 
ended their support prematurely, although CIDA 
managers had not identified that possibility as a 
risk. To ensure the expected results, CIDA 
responded on an emergency basis with a 
second project, to build and furnish classrooms. 
Progress reports on both projects have 
presented the status of activities and the results 
achieved to date, comparing them with the 
expected results. During our examination of 
these projects, we observed that the expected 
results were being achieved progressively. 
Classrooms were built or renovated with the 
involvement of the community and furniture was 
manufactured, both at considerably lower cost 
than expected. School programs were 
redesigned to enhance the community's 
interest, consequently increasing child 
participation. Local partners viewed the projects 
as successful and the government of Senegal 
expressed its intent to use it as a model 
approach for the country. 
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Our examination of another literacy project 
revealed deficiencies in risk management and 
results-based management. There was little 
attempt to identify internal risks; identified risks 
were ignored; and expected results remain 
vague. 


Project managers of intricate projects do 
not normally identify complexity as an internal 
risk. In this literacy development project with a 
budget of $15 million, the implementation 
suffered severe start-up problems. Our analysis 
indicates that part of the problem was the 
difficulty of dealing with a plan that called for 
numerous subcontractors to be managed by an 
on-site project management team of two 
Canadian executing agents and a recipient 
country project director. So far, results have 
been far below those expected. 


The Senegalese Ministry responsible 
conducted a risk assessment that identified a 
number of risks surrounding the project. One 
that it identified as significant was cost control 
over the operators (subcontractors). The project 
proposal, including the risk assessment, was 
made available to CIDA in February 1995. That 
same proposal produced the original costing of 
10,000 CFA francs per pupil. 


On 4 July 1996, CIDA signed a contribution 
agreement with the Canadian executing agent 
(CEA). The description of outputs in that 
contract stipulated that 200,000 people were to 
become literate over five years. The direct cost 
of that component was estimated at $5.6 million. 


———————— 
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In a development project with the 
objective of improving basic education 
facilities and access, CIDA contributed 
aid to build and furnish classrooms. 


Two weeks later, however, on 19 July 1996, 
CIDA’s representatives in Dakar reported on the 
operational planning mission conducted in early 
July. Their report estimated the costs per 
student at 22,772 CFA francs. It concluded that 
the estimate of output should be revised from 
200,000 people to 94,111. At the end of our 
audit, the project manager had not officially 
revised the output estimate. 

CIDA guidelines state that a significant 
decrease in output represents a cost increase. 
This cost increase should have been approved 
by the Vice-President. We noted that the project 
steering committee decided that the normal 
10-month course would be reduced to six 
months in order to meet the original output goal. 


This decision was made without any apparent 
study of its feasibility. 


As of July 1998, 18,000 students had 
passed the Ministry exam and 4,800 more were 
awaiting their results. The June 1998 Annual 
Project Progress Report notes that a downward 
revision should be made in the output goal, as 
the project is achieving results far below those 
expected for this module. We were told that 
negotiations are under way with the Senegalese 
government on the target number of people to 
become literate. CIDA terminated the 
contribution agreement with the CEA in 
June 1998 


With the implementation of results-based 
management, CIDA initiated new tools to 
measure results at the output, outcome and 
impact levels. For example, in the schooling 
projects that we audited, progress reports 
presented some actual results against expected 
results at the output and outcome levels. 
However, it was too early to report at the impact 
level. Two projects will end in 1998 and it will still 
be too soon to measure their real impact. There 
is no provision for follow-up on results at the 
project level after funding by CIDA ends. In 
addition, there is no provision at the country 
program level for gathering the basic 
information necessary to measure the long-term 
performance of CIDA's intervention. 
Consequently, there will be no assessment of 
the overall impact at either the project level or 
the country program level. 


Vietnam 


In 1994, CIDA established a Country 
Development Policy Framework for Vietnam that 
set out the goals and objectives of Canada’s 
ODA program, a strategy for achieving them, 
and the expected results. At the time, CIDA had 
limited operating experience in Vietnam. 


Compared with other bilateral donors, 
Canada’s financial commitment in Vietnam is in 
the middle range, with a bilateral country 
program of $15 million per year. Large-scale 
infrastructure and social development projects 


are beyond the scope of Canadian ODA 
resources. Accordingly, CIDA's strategy is to 
enhance Vietnam’s own capacity to address 
development objectives. It does this through a 
policy that supports the building of capacity and 
systems of governance. CIDA’s Country 
Development Policy Framework correctly 
recognized that the capacity-building approach 
presents certain risks to the achievement of 
expected results; and it recognized that 
individual projects are also vulnerable to these 
risks. CIDA also developed performance 


indicators for use in monitoring and evaluating 
achievements against expected results. 
Continuous monitoring and evaluation by a 
corporate monitoring committee were to be an 
integral part of the strategy. The committee was 
to meet regularly to review progress toward 
achievement of results, review constraints or 
problem areas, assess solutions, and share 
lessons learned. An evaluation of the 
Framework was called for in the fourth year of 
its implementation. 
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In Vietnam we examined six projects. Four, 
involving institution strengthening and capacity 
building, had planned expenditures totalling 
$38 million over their initial phase. Three of the 
four were experiencing some problems, from 
manageable to serious. The risk that these 
problems would occur had in some cases been 
identified earlier, but there was no specific plan 
for dealing with them. For example, from the 
outset and more than a year into the projects, 
CIDA and the country partner did not share a 
common understanding of what CIDA meant by 
capacity development and what results were 
expected from the projects. CIDA 
underestimated the considerable effort and 
resources required to reach a common 
understanding with its Vietnamese project 
partners. Early expenditures by CIDA had 
limited value because the country partner was 
not able to absorb the information provided by 
consultants from Canada. 


As a consequence, in another project 
(currently in the planning stage) CIDA mounted 
a particular effort to ensure that all parties 
understood the purpose of the project and the 
development result that was expected. CIDA 
was also looking at options for better oversight 


of the flow of funds, particularly in the early 
stages of the project. 
en 


Development of micro-credit systems 
in Vietnam. CIDA is helping to 
mobilize rural savings to finance 
small businesses. 


_ Assumptions about conditions for 
sustainability need to be re-examined regularly 
in such projects. Their very nature means that 
the country partner does not have the technical 
and managerial capability at the outset to 
sustain the project. Another possible indicator of 
project sustainability is the level of financial 
commitment and participation by local partners. 
In some projects, CIDA provided most of the 
financing and had to encourage local 
participation by providing per diem funding or 
special equipment. CIDA managers pointed out 
that they had little choice but to use such 


practices because other donors commonly 
provide these kinds of incentive. This raises 
questions about the potential sustainability of the 
projects. 


The review of the country program’s 
implementation was carried out in the fourth 
year, as the Bilateral Branch had planned. The 


‘draft review report noted that the corporate 


monitoring committee had met only a few times 
at the start of the program and was no longer 
active. It provided an update of the current 
development context, and discussed lessons 
learned from the experience of the past four 
years. It also led to a refining of CIDA’s 
development objectives. However, it did not 
review progress against the expected results set 
out in the original development framework. 


In our view, this was a missed opportunity. 
Having made the effort to articulate a set of 
expected results and possible indicators for 
measuring and reporting progress, CIDA would 
have benefited by using them to compare 
whether actual progress to date met its 
expectations, whether the results it had 
expected were realistic, whether the 
performance indicators were practical, and 
whether refinements were needed. 


(continued from page 21-17) 


had been no study, as called for by CIDA 
policy, to establish the amount of the net 
proceeds to be placed in the fund; the 
recipient country consistently withheld the 
full 7.5 percent. There was no accounting 
for how these funds had been spent, as we 
would have expected in the absence of 
such a study. Over the last three years, 
these charges averaged $930,000 per year. 
Given the potential for problems of this 
nature, CIDA will need to ensure that it 
applies its policy on payments to 
counterpart funds more consistently. 


21.57. Where a transaction results in a 
payment into a counterpart fund of less 
than the full amount paid by CIDA, 
CIDA project managers should 
document the rationale for the 
transaction. 


CIDA’s response: CIDA will strengthen its 
procedures to include the need to 
document adequately the rationale for this 
kind of transaction in the management of 
counterpart fund projects. 


Institutionalizing Lessons 
Learned 


CIDA has made considerable progress 


21.58 In 1993 we were concerned that 
CIDA did not have an organizational 
“learning culture”. We urged the Agency 
to institutionalize lessons learned at the 
project and country levels. In response to 
our comments, CIDA agreed to initiate a 
simple and clearly understood mechanism 
that would allow lessons learned to be 
captured and accessed. 
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21.59 CIDA undertook formal and 
informal steps to strengthen its learning 
culture. Lessons learned are an integral 
part of the Policy for Performance Review 
and the Results-Based Management Policy 
Statement. They are one of the key 
elements of the Annual Project Progress 
Reports and the end-of-project reports, 
and are incorporated into branch-level 
achievement reports. Lessons learned are 
being synthesized as part of each 
corporate review carried out by the 
Performance Review Branch. The Agency 
has set up various vehicles to gather and 
disseminate lessons learned. Examples of 
these vehicles are the President’s Forum 
on best practices, and informal networks 
such as the results-based management 
practitioners’ network and others that 
discuss areas like Women in 
Development, and Environment. 


21.60 CIDA has made considerable 
progress in incorporating the concept of 
lessons learned into the results-based 
management approach. The system for 
capturing lessons learned relies mainly on 
the APPR. The quality of information on 
lessons learned varies considerably among 
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APPRs. CIDA needs to do more to 
consolidate, analyze and disseminate this 
information to make it useful to staff. 
Otherwise, the risk is that project staff will 
not see the exercise as useful and its value 
to the organization and to performance 
reporting will be lost. 


CIDA has made 
considerable progress 
in incorporating the 
concept of lessons 
learned into the 
results-based 
management 
approach. 


Audit and Evaluation 


Further effort is needed on 
performance review 


21.61 In July 1994, CIDA introduced 
its Policy for Performance Review. This 
policy established a framework for the 
audit and evaluation functions and has had 
the effect of creating a closer connection 
between the Performance Review Division 
and operational branches at CIDA. The 
Policy also called for a Performance 
Review Committee with membership 
drawn from inside and outside CIDA, in 
order to demonstrate an openness to 
renewing performance review and to 
benefit from the experience of other 
departments. However, this committee 
met only twice, in 1995. We were 
informed that the Executive Committee 


agreement with the recipient country. 


The counterpart fund in Senegal involved the sale of Canadian sulphur. There is a world market 
for sulphur. We found in at least three shipments that the buyer in Senegal would not agree to pay 
the price per ton that the Canadian supplier had quoted, insisting instead on paying no more than 
the going international price. CIDA financed the transactions at the higher price, and allowed the 
buyer to deposit the lower amount to the counterpart fund. As a result, in the three transactions 
CIDA paid Canadian suppliers a total of US $844,000 above market rates in order to maintain the 
viability of the counterpart fund. This was contrary to the intent of CIDA policy and to the 


In answer to a letter from a Canadian supplier complaining about the bidding procedures for this 
project, CIDA’s President responded in August 1993: 


... In the case of CIDA’s lines of credit with recipient countries, the final decision to 
contract always rests with the buyer who is required to reimburse his government an 


Exhibit 21.7 


Problems in Control Over the 
Flow of Money Into a 
Counterpart Fund 


equivalent value in local currency to that paid by CIDA to the supplier in Canada. If the 
buyer does not agree with the price, he/she simply refuses to enter into contract and the 
tender is cancelled.... 


We did not find appropriate written authority for CIDA to act contrary to these principles. 
Officials told us that there were no alternative products to provide funding for the counterpart fund 
had the sales not gone through. As this fund was necessary to pay for the Senegalese portion of 
many of CIDA’s projects in that country, lack of funds would have jeopardized nearly all those 
projects. This was recognized by CIDA and we were informed that corrective action is being 
taken. No substantiated explanation was on file for the higher-than-world-market prices quoted by 
Canadian firms. 


— SS 
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The Agency needs to 
maintain its 
momentum toward 
implementing 
management for 
results. 


performs this role now. In 1997, the 
Performance Review Division obtained 
Branch status and began to report directly 
to the President of CIDA, and its 
Director-General became a member of the 
Executive Committee. 


21.62 In 1995, CIDA reported to 
Parliament that it planned to conduct 
performance reviews of the six ODA 
programming priorities over the next 
three years. However, significant slippage 
has occurred. To date, the Performance 
Review Branch has undertaken three 
reviews covering two of the priorities. The 
Branch indicated that these reviews had 
proved more complex and costly than 
planned. The reviews also incorporated a 
more comprehensive approach, designed 
to identify implications for policy 
development in the ODA priority areas as 
well as lessons learned from individual 
projects. In addition, the Branch was 
assigned the lead role in the development 
and implementation of results-based 
management across the Agency. The 
remaining reviews of the ODA priorities 
have now been rescheduled over the next 
few years. 


21.63 We noted that the Policy for 
Performance Review did not address 
concerns we expressed in 1993 about 
project evaluations (now called 
management-led operational reviews). We 
felt then that there was a need to guard 
against a perceived lack of objectivity, 
mainly because project evaluations were 
commissioned by the Geographic Program 
staff. There was no ongoing quality 
assurance function in CIDA that examined 
whether minimum evaluation standards 
had been met. 


21.64 The Branch has since undertaken 
several initiatives to help ensure more 
consistent quality of operational reviews. 
It has encouraged use of the Framework of 
Results and Key Success Factors as the 
basis for these reviews, and has 
established a standing offer list that 
Geographic Program staff can use to 


contract for the services of performance 
reviewers. In addition, on request the 
Branch provides guidance on the conduct 
of operational reviews. 


21.65 These initiatives represent 
progress, but they still do not address our 
concerns. We believe that the Agency 
needs to monitor at the corporate level 
whether expected quality standards are 
being met. 


21.66 The Performance Review 
Branch should selectively review the 
quality of operational reviews 
conducted by the geographic branches 
as part of its regular audits and 
evaluations. 


CIDA’s response: CIDA will, on a 
selective basis, review the quality of the 
operational reviews conducted by the 
geographic branches as one of the lines of 
inquiry in its audits and evaluations. 


Conclusion 


21.67 The Agency is now well into the 
renewal program it began in 1994. The 
framework for results-based management 
along with supporting policies and 
guidelines have been developed and 
communicated to CIDA’s staff and 
development partners. We concluded that 
CIDA’s actions have addressed the main 
concerns we raised in our 1993 Report. 
However, while a more determined focus 
on results is evident at the levels of 
country program and project delivery, 
progress in implementing results-based 
management has been uneven. 


21.68 The Agency needs to maintain its 
momentum toward implementing 
management for results. In focussing on 
results, it needs to take a harder look at 
whether country programs or projects 
should be continued, scaled back or 
rethought when expected development 
results are at risk because of changes in 
critical underlying assumptions, absence 
of essential conditions for sustainability, 
or other factors that could have an adverse 
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impact. The Agency also needs to monitor 
projects in a cost-effective way after their 

completion to determine whether expected 
results have been achieved. 


21.69 The Agency has developed a 
performance reporting system that starts at 
the project level and ends with its 
Performance Report to Parliament. Both 
internal and external reporting can be 
improved by including information on 
expected as well as actual results at the 
project and country levels, and by 
presenting a more balanced picture of 
accomplishments, difficulties and 
challenges in delivering development 
assistance. 
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21.70 All donors recognize that it is 
difficult to measure development results at 
the country or program level. However, 
the Development Assistance Committee 
of the OECD is developing a working set 
of core indicators to measure development 
progress at a level higher than that of the 
individual projects. CIDA has contributed 
to the development of these indicators, 
which are still being refined and tested. As 
CIDA gains more experience in 
results-based management, indicators such 
as these or others that may emerge could 
provide it with a means of reporting 
program results to Parliament. 


Report of the Auditor General of Canada — December 1998 21-25 


Canadian International Development Agency — Geographic Programs 


ty About the Audit 


In 1994, CIDA and the Office of the Auditor General agreed that we would conduct a phased follow-up of our 
1993 audit of CIDA’s bilateral programs for official development assistance. The work would focus on actions 
taken by CIDA to address the main concerns raised in our 1993 Report. This audit represents the third and 
final phase of the “phased follow-up” approach. 


Objectives 
Our objectives were: 


e ‘Toassess the extent to which CIDA’s actions in implementing a results-based management approach have 
satisfactorily resolved the main concerns raised in our 1993 Report. 


e To review the quality of performance reporting on development results. 


Scope and Approach 


Our audit continued to focus on CIDA’s accountability to Parliament for managing for results. We reviewed 
the way CIDA plans and manages its bilateral programs and projects, and measures and reports their results. 


We used a combined country- and project-based approach to conduct this audit. CIDA identified four country 
programs: Bolivia, Estonia, Senegal and Vietnam. CIDA believes that these best demonstrate the 
improvements gained by implementing results-based management in the bilateral development aid program. 
We added Bangladesh to provide a basis for comparison with our 1993 audit. In each of the identified 
countries, we selected a sample of projects for audit. 


We conducted field work at Canadian missions and project sites in the selected countries. It included review 
of on-site documentation, as well as consultations with stakeholders such as Canadian and local executing 
agents, with host country officials and with representatives from other international donor organizations. 


At CIDA’s headquarters, we examined the process and the underlying reasons for selecting country programs 
and projects and the process for measuring and reporting their results. We also assessed the extent to which 
the key concerns raised in our 1993 Report had been satisfactorily addressed. 


Our work did not extend to comparing CIDA’s performance with that of other organizations providing 
bilateral aid. 


Criteria 


The criteria used at the country program level and at the project level were based on criteria proposed in our 
| 1996 Report: 


Country level 

Results Achievement 

e Well-defined objectives that can be reasonably achieved are actively pursued. 
Risk Management 


e Risks associated with the program and related activities are assessed and managed. 
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Performance Measurement 


e Relevant information on performance is obtained and used. 

e The extent to which programs and activities are meeting the Agency’s performance expectations is 
evaluated and understood. 

Project level 


Results Achievement 


e Well-defined objectives that can be reasonably achieved are actively pursued. 

e Projects are followed up to determine whether project partners are achieving (or are likely to achieve) 
planned results. 

Risk Management 

e Risks associated with the projects and related activities are assessed and managed. 


Sustainability 


e There is follow-up to assess whether project results are likely to provide sustainable benefits after CIDA’s 
direct funding comes to an end. 

Performance Management 

e Relevant information on performance is obtained and used. 


e The extent to which projects and activities are meeting performance expectations is evaluated and 
understood. 


e Timely action is taken to improve project performance. 


e Periodic reports on the potential for results and corrective action taken where appropriate are presented to 
CIDA’s senior management. 


e CIDA is providing assurance that money has not been expended for purposes other than those for which it 
was appropriated. 


Institutional level 


Learning Organization 


e CIDA has created an action-oriented organizational environment that promotes learning by doing, an 
innovative style of management and an openness to differing viewpoints. 


Audit Team 


Assistant Auditor General: David Rattray 
Principal: John Hitchinson 
Director: Manfred Kuhnapfel 


Johane Garneau 
Tina Guthrie 
Jacques Marquis 
Paul Morse 
Barry Neilson 
Jaak Vanker 
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The Federal Science and 
Technology Strategy 


A Review of Progress 


Main Points 


22.1 It has been nearly three years since the government released Science and Technology for the New Century 
—A Federal Strategy. The Strategy outlined three goals to which science and technology (S&T) resources were to 
be directed. It recognized the need for better management of S&T activities and provided a set of principles to 
improve their management in departments and agencies. 


22.2 In this follow-up to our 1994 audit of the federal S&T effort, we found that progress in establishing the 
elements required to improve its management has, overall, been slow. The government has not acted fully on 
some of the commitments it made in the Strategy. As a result, the management regime set up to oversee the 
federal S&T effort is not yet working as intended by the government — that is, ensuring that priorities are clear, 
that activities are co-ordinated, and that performance is reported on fully. 


22.3 In the departments we examined, progress in acting on the Strategy’s seven operating principles has 
varied considerably. However, even the less advanced departments have moved in the direction desired by the 
government. More remains to be done, and our follow-up identified three areas that require special attention: 
mission-driven, results-based research; scientific excellence; and the use of partnerships. 


22.4 Some results of the Strategy are now emerging, and we believe it may be time for the government to 
clearly set out what remains to be done — what is next on the agenda. Along with sufficient leadership and 
ongoing parliamentary oversight, this would help to ensure that the current Strategy does not become, like similar 
efforts over the last 30 years, one more missed opportunity. 
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Introduction 


22.5 It is well recognized that science 
and technology contribute to the country’s 
ability to generate sustainable 
employment and economic growth. But 
science and technology go beyond 
contributing to economic growth; they 
support our quality of life. 


22.6 The federal government invests 
significant resources in science and 
technology (S&T) activities. In 1997-98, 
departments and agencies spent an 
estimated $5.45 billion on S&T. The 
federal government also encourages 
research and development in the private 
sector through tax incentives that 
represent more than $1 billion annually. 
Those amounts do not include funding for 
the Canada Foundation for Innovation, 
which in 1997 received a federal grant of 
$800 million to provide new infrastructure 
support to university and medical research 
and development. 


22.7 Given the importance of S&T to 


our country, and the substantial 
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Technology Activities, The Management 
of Scientific Personnel in Federal 
Research Establishments, and Income Tax 
Incentives for Research and Development. 


22.9 In its 1994 Budget several 
months before the publication of those 
chapters, the government had announced a 
fundamental review of its S&T effort 
aimed at giving Canadians a federal 
strategy with “real priorities, real 
direction, and a real review of results”. 


22.10 Our 1994 report documented the 
failure of previous such efforts in the last 
30 years. Indeed, we said that there had 
been much activity but few results. In our 
view, some of this lack of progress could 
be attributed to a lack of overall 
government-wide leadership, direction 
and accountability. Accordingly, we 
presented lessons learned from past efforts 
for the government to consider in its 


-announced review. 


22.11 ~_—‘In brief, our observations were 
that the federal government needed to 
provide: 


In 1997-98, 
departments and 
agencies spent an 


: e clear priorities and clear direction for j ill 
investment involved, the proper ane estimated $5.45 billion 
baie its S&T activities; : 
management of S&T activities is of on science and 
paramount importance. In our view, there e clear performance expectations and 
; technology (S&T) 
are two fundamental questions to ask action plans; 
about the government’s S&T Arent iviti 
é e effective co-ordination and activities. 

performance: a hoe 

oversight; 

« To what extent has the government + leadership that transcends 
clearly established what it wants to departmental mandates; and 
achieve with its S&T activities? 
e better information for Parliament and 
* To what extent is it managing S&T the public on S&T activities and 

activities in a way that maximizes value performance. 
for money? 

22.12 In 1995, the government 
Previous audit findings launched Program Review, a 

government-wide initiative aimed at 
22.8 In an attempt to answer those two __ rethinking and realigning government 
questions, in 1994 we conducted a sectoral _—_ programs in light of fiscal restraints. 
audit of federal science and technology Program Review had a major impact on 
activities. The results were presented in some science and technology departments. 
four interrelated chapters of our 1994 For some departments it also constituted a 
Report: Overall Management of Federal priority-setting exercise involving senior 
Science and Technology Activities, officials and ministers. During the same 
Management of Departmental Science and _ period, the government started to revise its 
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The 1996 S&T Strategy 
addressed certain 
government-wide 
concerns we had 
raised in our 1994 
Report. 


In our view, the 
Strategy left some 
“unfinished” business. 


Expenditure Management System (EMS) 
to improve the information provided to 
Parliament on expenditure management, 
planning and performance. An integral 
element of the EMS was the Treasury 
Board’s requirement that departmental 
Business Plans be submitted to it for 
review. 


22.13 In March 1996, the government 
published Science and Technology for the 
New Century — a Federal Strategy (the 
Strategy). Through the Strategy, the 
government set out three goals for the 
federal investment in S&T: 


e to ensure that Canada is among the 
best in the world in applying and 
commercializing S&T for sustainable job 
creation and economic growth; 


e to ensure that Canada applies S&T to 
improve the quality of life for our citizens 
through the creation of fulfilling jobs and 
through the most effective social, 
environmental and health care programs 
in the world; and 


e tocreate in Canada world centres of 
excellence in scientific discovery, to build 
a broad base of scientific enquiry, to foster 
Canadian participation in all major fields 
of science and technology, and to ensure 
that new knowledge can be acquired and 
disseminated widely from Canadian 
sources and from around the world. 


22.14 The Strategy provided direction 
to departments and agencies and set out 
the elements of a federal governance 
system for their S&T activities through 
which to achieve these goals. The Strategy 
document was accompanied by action 
plans from departments and agencies 
involved in S&T. The government also 
released its Framework for the Human 
Resources Management of Federal 
Science and Technology Community. 


22.15 Ina follow-up chapter published 
in September 1996, we welcomed the 
S&T Strategy, noting that it addressed 


certain government-wide concerns we had 
raised in our 1994 Report. We found that 
science-based departments had made 
plans to focus their science and 
technology activities and set priorities. 
The Strategy provided direction to 
departments and agencies in the form of 
operating principles for conducting many 
aspects of their science and technology 
activities. We observed that departments 
and agencies had recognized the 
advantages of working more closely 
together. Various initiatives were also 
under way to improve the assessment of 
results and impacts of federal science and 
technology activities, and to report on the 
growth of Canada’s knowledge-based 
economy. 


22.16 However, in our view the 
Strategy had left some “unfinished” 
business. For example, even though a 
mechanism had been established to make 
recommendations to Cabinet on S&T 
priorities, government-wide priorities had 
not been enunciated. We noted the critical 
need for the government to monitor the 
Strategy’s implementation and to devote 
considerable attention to establishing 
results-oriented, time-phased 
implementation plans. This would help 
ensure that the direction provided to 
departments and agencies would become 
an integral part of day-to-day management 
in science-based organizations. We noted 
that accomplishing this would require 
leadership and perseverance at all levels 
of government. We also said that 
parliamentary oversight was key to 
ensuring that it happened. 


Focus of the follow-up 


22.17 Given the importance of the 
Strategy and the fact that two years have 
passed since our last follow-up, we 
believe the time is right to build on our 
earlier work with a review of the progress 
made in implementing the S&T Strategy. 
This follow-up chapter provides our 
assessment of progress to date in: 
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¢ meeting the Strategy commitments 
to establish new institutions and 
mechanisms for management; and 


¢ acting on the operating principles set 
out for departments by the government. 


22.18 We looked at the progress made 
by four departments with significant direct 
involvement in S&T: Agriculture and 
Agri-Food Canada, Fisheries and Oceans, 
Environment Canada and Natural 
Resources Canada. As a case study (see 
page 22—14), we also assessed the 
management of climate change science 
activities — looking specifically at the 
extent to which it manifests the principal 
commitments of the Strategy. Given the 
limited sample size, the results discussed 
in this chapter should not be generalized 
to all science-based departments and 
agencies. 


22.19 Details on the objectives, scope 
and approach of our work are included in 
About the Follow-up at the end of this 
chapter. 


Observations 


Important Challenges Still Lie 
Ahead 


22.20 In the course of its science and 
technology review, the federal government 
received a number of suggestions for 
changing the structure and processes of its 
decision making on science and 
technology. The government intended to 
respond by creating and reorganizing 
institutions and mechanisms to improve 
the governance of S&T. 


22.21 In the Strategy, the government 
recognized that to be a more effective 
partner in the country’s innovation system, 
it needed to get its own house in order. It 
concluded that achieving greater 
coherence while preserving flexibility, 
responsiveness and ministerial 
accountability would require a more 
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rigorous collective review of priorities and 
greater co-ordination of activities. 


22.22 Accordingly, the Cabinet 
Committee on Economic Union (CCEU) 
was mandated to annually review the 
performance of federal S&T activities and 
recommend priorities to Cabinet. To 
facilitate its review of S&T priorities, the 
CCEU would receive advice from a new 
body, the Advisory Council on Science 
and Technology, which replaced the 
National Advisory Board on Science and 


Technology. 
a The government 


recognized that to be a 
more effective partner 
in the country’s 
innovation system, it 
needed to get its own 
house in order. 


22.23 The Strategy also stated that 
improving top-level advisory and 
decision-making structures was not 
enough to ensure that the federal 
government’s substantial investment in 
S&T would yield better results. The 
government recognized that to improve 
the management of its investment, it 
needed more co-ordination of S&T 
activities among federal agencies and 
collaboration on major horizontal issues 
— those that cut across departmental and 
agency boundaries. The Minister of 
Industry would lead the co-ordination of 
S&T policy and strategies across the 
federal government. This effort would be 
supported by another new body, the 
Council of Science and Technology 
Advisors. 


22.24 The intended interactions among 
the new and renewed institutions are 
presented in Exhibit 22.1. 


22.25 The Strategy contained other 
specific commitments: 


e development of new measures to 
increase the accountability of individual 
departments and agencies for the 
management of their S&T; 


¢ implementation of a Framework for 
the Human Resources Management of the 
Federal S&T Community; 


e creation of anew S&T information 
system; and 


¢ new direction for intergovernmental 
co-ordination and co-operation. 
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22.26 Two years after our previous done so far, and indicate the remaining 
follow-up, we expected to find the new challenges that must be met if the system 
institutions and mechanisms in place and is to work well. 


to see indications that they would soon be 


working well. Overall, however, we found Elements of the management regime are 


slow progress in establishing the new im plate 

management system, and some 22.27. The Advisory Council on Science 
commitments have not been addressed and Technology was created in July 1996. 
adequately. As a result, the system is not Comprising 11 members, it is chaired by 
yet working as the government intended the Minister of Industry and supported by 
— that is, ensuring that the federal S&T a secretariat in Industry Canada. Its 
management system has clear priorities, mandate is “to review the nation’s 
co-ordinated efforts, and full reporting on performance in S&T, identify emerging 
performance. In this section, we describe issues and advise on a forward-looking 


our specific concerns about what has been _— agenda”. Initially, it intended to work on 
Exhibit 22.1 


Management Regime Intended for Federal 
Science and Technology 


Cabinet Committee on 
Economic Union 
reporting to 


avcory Coanclicn — annual review of federal 


: science and technolo i 
Science and Technology ‘ f by proses ie 
(ACST) AS SEC Perouse Council of Science and 
— recommendations to Cabinet give mandate to Technology Advisors 
nae on science and technology SSS 
grees a priorities 
collaboration 
a ; support 
chais Minister of Industry Industry Canada’s cracker 
Ao Science and 
— co-ordination of science and Technology Secretariat 

technology policy and strategies 

— across the federal government 
Ministers of 
science-based ; 
support 
departments and | : 
aeenties function 
: ADM Committes on Seiencene: Secretary of State (Science, Research 
— retain the Technology and Development) 

authority and ; i ae eye 2 a i 
een (senior officials of science-based — support for the Minister of Industry chair 
for S&T related | . departmcnmiandiapencs) — regular interface between 
to their mandates | | — implementation of science and government and Council of Science 

technology Strategy and Technology Advisors 

— action on cross-government science 
id Mayo and technology issues 
collaboration collaboration 
A 
drawn 
from 
P, TECEIVE gi from Advisory committees in science-based 


| departments and agencies m 
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three specific issues: human resource 
challenges as Canada moves to become a 
more knowledge-intensive society; the 
potential for commercializing Canadian 
research; and development of a more 
coherent and publicly accepted vision for 
the future of science and technology in 
Canada. The Council was to also advise 
the Cabinet Committee on Economic 
Union (CCEU) on priorities for science 
and technology. Its first meeting with the 
CCEU (and the only one to date) was in 
December 1997. 


22.28 The Council of Science and 
Technology Advisors was created in May 
1998 and first met in October 1998. Its 
22 members are drawn mostly from the 
advisory committees of science-based 
departments and agencies. The Council is 
chaired by the Secretary of State for 
Science, Research and Development, and 
will report to the CCEU. Supported by an 
Industry Canada secretariat, the 
Committee will look at S&T issues 
internal to government that require 
government-wide strategic attention. The 
CCEU has asked the Council to begin 
looking at the federal government’s roles 
in conducting S&T and at its ability to 
fulfil those roles; and to develop a set of 
guidelines on the use of scientific advice 
in government decision making. 


Committees have not reached full 
potential 


22.29 Although the Strategy in 1996 
indicated the need for more external 
support for the co-ordination of S&T 
activities, it was not until May 1998 that 
the Council of Science and Technology 
Advisors was created. We are concerned 
that by the time it eventually makes 
recommendations, opportunities to address 
important horizontal issues will have been 
significantly delayed. For example, as 
already noted, the Council has been 
mandated by the CCEU to look at the 
government’s roles in conducting S&T 
activities and at its ability to fulfil those 
roles. In our opinion, this is an issue that 


‘policy issues; and 
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needs to be tackled as soon as possible 
because of its impact on the way 
science-based departments and agencies 
conduct their business. 


22.30 The Assistant Deputy Minister 
Committee on Science and Technology 
(ADM Committee) is composed of senior 
officials at the assistant deputy minister 
level from science-based departments and 
agencies. It provides the government with 
the capacity to deliver on its collective 
responsibility for policy co-ordination and 
management of federal S&T. In 1996, we 
asked what specific role the Committee 
would play in the implementation of the 
Strategy. In 1997, the Committee’s terms 
of reference set out its revised mandate: 


Overall, we found slow 
progress in 
establishing the new 
management system, 
and some 
commitments have not 
been addressed 
adequately. 


e toimplement the cross-government 
commitments made in the Strategy; 


e to develop proposals and advice to 
the government on key horizontal S&T 


e to provide a forum for 
interdepartmental consultation on S&T 
policy and program directions, sharing of 
information, and co-ordination of efforts 
and initiatives across the federal S&T 
system. 


In addition, the new mandate given to the 
Council of Science and Technology 
Advisors calls for close collaboration with 
the ADM Committee. 


22.31 Although it does not have a direct 
link to Cabinet, the ADM Committee 
could play an important role in the 
management system. Under the Strategy, 
however, it has no formal responsibilities 
for co-ordination — a crucial gap, in 
particular since the Council of Science 
and Technology Advisors was not created 
until May 1998. 


22.32 Given its mandate, we expected 
that the ADM Committee would provide a 
forum for discussing issues of common 
concern as well as a vehicle for getting 
things done. We found that the Committee 
has discussed many issues and created 
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The government needs 
to clarify what it 
expects the Assistant 
Deputy Ministers 
Committee to 
accomplish. 


To make the bodies 
involved work well 
together, three 
ingredients are 
essential: a work plan 
for each of them, 
regular interaction and 
an ongoing exchange 
of information. 


working groups to deal with them but few 
initiatives of these working groups have 
reached the implementation stage. One 
notable exception was the Committee 
input to the publication in 1997 of the first 
annual report on S&T activities. 


22.33 We found that some issues have 
been “ongoing” for a long time. For 
example, a framework for international 
S&T was to be established to meet one of 
the Strategy commitments: the 
development of explicit plans to promote 
international S&T collaboration for the 
benefit of Canadian firms. The final 
framework was to be completed in March 
1997 but, more than a year and a half 
later, it is still not finalized. Revising the 
Intellectual Property Policy was another 
commitment in the Strategy. A second 
round of consultations is now under way 
but, after two years, nothing has been 
finalized. 


22.34 In our opinion, since the ADM 
Committee is an essential part of an 
effective co-ordination system, the 
government needs to clarify what it 
expects the Committee to accomplish. 
Beyond this, to be effective the 
Committee requires strong, consistent 
leadership and secretariat support. 
Members of the ADM Committee need to 
consider ways to ensure that it fulfils its 
potential. 


How committees will work together is 
unclear 


22.35 We had expected the government 
to ensure that the respective mandates and 
work plans of committees were integrated 
so that the S&T management system 
would operate as effectively as possible. 


22.36 While the Council of Science and 
Technology Advisors was to provide an 
external perspective on internal federal 
management of S&T, the Advisory 
Council on Science and Technology 
(ACST) would be expected to provide its 
views on broader issues, beyond internal 


management — for example, science and 
technology and youth. However, it would 
be reasonable to assume that the two 
bodies would complement each other’s 
work on specific issues. 


22.37 Since it is important that these 
two bodies work well together and with 
the ADM Committee, we would expect to 
see Clearly how they would interact. In our 
view, three ingredients are essential: a 
work plan for each of them, regular 
interaction and an ongoing exchange of 
information. 


22.38 A work plan is essential to 
identify what needs to be done, when and 
by whom. It would also help in tracking 
progress on issues and in holding officials 
accountable for results. The ACST has 
identified certain issues to address. The 
recent creation of two expert panels on 
skills in knowledge-intensive industrial 
sectors and the commercialization of 
university research provides a better idea 
of how two of these issues will be dealt 
with. However, there is no further 
indication of how and whether the ACST 
intends to work on the third issue 
identified (the development of a publicly 
accepted vision for the future of S&T in 
Canada) or whether any other issues will 
be looked at while the panels produce 
their reports. The Council of Science and 
Technology Advisors will also need to set 
out its goals and a work plan. At the end 
of our review, we were informed that a 
work plan was being drafted. 


22.39 In February 1997, the ADM 
Committee decided that it needed a work 
plan to help implement the 
government-wide commitments made in 
the S&T Strategy. The work plan was to 
be based on the deliverables indicated in 
the Strategy. The ADM Committee also 
believed that it needed to prioritize the 
deliverables and identify those it would 
address over the next year. However, 
officials have informed us that the ADM 
Committee never produced such a work 
plan. 
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22.40 We found some links among the 
three bodies. For example, the ADM 
Committee was informed regularly of the 
work done by the ACST. The terms of 
reference for the Council of Science and 
Technology Advisors state that it will 
work in collaboration with the ADM 
Committee on horizontal issues; they are 
also supposed to meet together at least 
once a year. As well, a Deputy Chair of 
the ACST (to be appointed) will serve as 
an ex-officio member of the Council of 
Science and Technology Advisors. 
However, since this Council has just been 
created, we cannot comment on how well 
integrated their respective work plans will 
be. 


22.41 ‘In our view, Industry Canada has 
a key role to play in making all of this 
happen. We encourage it to ensure that 
appropriate linkages exist among the three 
bodies. 


Need to move beyond co-ordination to 
collective action 


22.42 In our 1996 follow-up report we 
said that the government’s science and 
technology review had made 
science-based organizations more aware 
that they needed closer ties and improved 
relationships to tackle certain issues 
successfully. New initiatives mentioned at 
that time are now well under way, like the 
Federal Partners in Technology Transfer 
that involves 14 departments. Another is 
the Memorandum of Understanding 
(MOU) to foster collaboration and 
co-ordination in S&T for sustainable 
development, signed by all four 
departments we examined. It was recently 
renewed and now includes Health Canada. 
Other initiatives have followed. 


22.43 However, more co-ordination 
does not necessarily lead to better 
management of horizontal issues. In our 
view, effective management of any 
important horizontal issue needs to 
include an action plan that orchestrates the 
relevant activities of departments and 
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other players involved to achieve 
agreed-upon objectives; an accountability 
framework that transcends departmental 
Jurisdictions; and a joint reporting 
mechanism to track results in relation to 
objectives. While we did not examine 
co-ordination among departments per se, 
we looked at the management of climate 
change science to see whether there was 
evidence of good co-ordination and 
collaboration in practice. 


22.44 While the MOU in S&T has been 
an important step in co-ordinating science 
efforts in climate change, it does not fulfil 
our expectation for joint goal setting and 
research planning and a common 
management framework. Nevertheless, 
there are signs that the federal government 
is moving beyond co-ordination to 
collective action, with the adoption of a 
new management framework. The case 
study on page 22—14 contains our 


More co-ordination 
does not necessarily 
lead to better 
management of 
horizontal issues. 


. findings. It appears that the renewed 


National Biotechnology Strategy is also 
adopting a promising approach, although 
it was not part of our review. We hope that 
those experiences will lay a new 
foundation for the management of other 
horizontal issues. 


Review of priorities is incomplete 


22.45 As noted in Minding Our Future, 
the first annual report on federal S&T 
activities, “The Government outlined its 
priorities in the Speech from the Throne. 
Investing in knowledge and creativity and 
developing the workforce for the 21st 
century are key aspects of the efforts to 
build a stronger Canada. These objectives 
will also shape the federal S&T effort in 
the coming years.” Those general 
objectives, along with the goals set out in 
the Strategy (sustainable job creation and 
economic growth; improved quality of 
life; and advancement of knowledge), 
have been translated into spending 
decisions over the past two years. 


22.46 The government has made 
explicit choices in directing new S&T 


funds to non-departmental entities like the (continued on page 22-17) 
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Management of Federal Activities in Climate Change Science 


The purpose of this case study was to determine whether the federal government's management 
of its climate change science activities reflects the key commitments and principles of Science and 
Technology for the New Century — A Federal Strategy. \n other words, we wanted to know 
whether the federal government was putting the Strategy into practice in this area. 


The management of climate change science to this point does not reflect fully the intent of the 
commitments made in the Strategy. However, efforts are under way to better manage climate 
change science. We hope that these efforts will lay a new foundation for the management of other 


horizontal issues. 


Introduction 


Canadians have been participating actively for 
many years in researching and monitoring 
global climate change. In conjunction with a 
number of stakeholders, federal government 
departments have prepared several reports 
stating that the effects of climate change on 
Canada are potentially serious. They note that 
the burning of fossil fuels, population growth, 
and changes in land use such as deforestation 
are producing global climate change, and that 
Canada is vulnerable to this change. Even with 
reductions in greenhouse gas emissions, 
changes in climate will continue and Canadians 
will have to adapt. 


Effects are expected on every region and 
sector, particularly agriculture, forestry and 
fisheries. Significant direct adverse effects could 
include changes in precipitation patterns leading 
to drier summers in the Prairies and central 
Canada, an increase in forest fires and insect 
infestations, changes in migration patterns of 
fish stocks, coastal flooding from rising sea 
levels, extensive thawing of permafrost in the 
north and more frequent severe weather 
events. While many of the effects of climate 
change are expected to be negative, there 
could be some potentially positive effects in 
some parts of Canada, such as milder winters 
and a longer growing season. 


Because climate change is a global problem, 
research on climate change requires 
international collaboration to generate scientific 


data on a global basis. A large component of 
Canadian research is linked to one or more of 
the three major programs developed by the 
international research community to address 
specific questions related to global change: the 
World Climate Research Program, the 
International Geosphere-Biosphere Program, 
and the International Human Dimensions 
Program. These programs have identified many 
key scientific problems that need to be 
addressed on a global scale, and have 
developed scientific rationale and plans to 
resolve these questions. This is the context in 
which the planning and implementation of 
Canadian research on climate change takes 
place. 


The Canadian research effort involves many 
players: the federal government, the provincial 
and territorial governments, the academic 
community, the private sector and community 
interest groups. The Canadian Climate Program 
Board, an advisory body composed of 
individuals from the federal and provincial 
governments, the academic community, the 
private sector and non-governmental 
organizations, co-ordinates climate-related 
activities in Canada. It provides advice and 
information on climate change and variability by 
communicating with a wide audience, including 
individuals and organizations involved in 
activities that contribute to the Canadian effort. 


What departments are doing 


Several federal departments are involved in 
research to better understand aspects of the 


climate system and determine the potential and 
measurable impacts of climate change on 
ecosystems. Our expectations for 
interdepartmental collaboration flow directly 
from the Strategy and are discussed under 
three broad areas: common goals, shared work 
and joint performance review and reporting. 


Common goals. Given the horizontal, 
integrated nature of climate change science, we 
expected that the efforts of individual 
departments would be guided by a common 
management framework. Such a framework 
would set out the objectives for a 
comprehensive federal effort that would include 
participation by relevant players outside the 
federal government. Determining research 
objectives in a global research endeavour such 
as climate change involves answering three 
questions: What knowledge do we need? 
Should Canada develop this knowledge? If so, 
who in Canada can do it? 


The introduction in 1995 of the Memorandum of 
Understanding (MOU) on S&T for sustainable 
development among the four natural resource 
departments led to the creation of a working 
group on Climate Change and Variability. This 
working group has been identifying the current 
state of knowledge, the research questions that 
need to be addressed, and priorities for action 
by the four resource departments. In 1998, 
Health Canada was added to this MOU. 


While the MOU has been an important step in 
co-ordinating science efforts, it does not fulfil 
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our expectation for joint goal-setting and 
research planning and a common management 
framework. 


In our view, it is important that the federal 
government formally identify corporate goals 
and plans for research into climate change 
through a comprehensive approach involving all 
relevant federal sectors and external 
stakeholders. Moreover, given the global nature 
of the issue and the need to link the Canadian 
effort with the international one, the goals and 
plans should identify the appropriate niches for 
Canadian contributions to the international 
research effort. Without common goals and 
plans, climate change science may simply be a 
collection of individual departmental programs 
or projects in the same broad area of scientific 
inquiry, rather than a deliberately orchestrated 
set of research activities aimed at resolving 
specific issues or problems. 


Shared work. The Strategy identifies the need 
to improve interdepartmental collaboration and 
co-operation in S&T activities in order to 
optimize the use of resources. For the 
management of climate change science, we 
expected to find an overarching structure that 
would link the research activities of various 
departments required to achieve common 
goals. That is, it would identify the research to 
be done and by whom, and by when. It would 
also estimate the cost of doing the research 
and identify the source of funding. 


We did not find an overarching structure to 
manage the research and the resource 
requirements. However, the existing 
mechanisms do enable the principal 
departments to work together and with partners 
outside the federal government. For example, 
Environment Canada has the Canadian 
Climate Research Network, which has 
mobilized the academic community across 
Canada and helped to co-ordinate aspects of 
government/university efforts. Natural 
Resources Canada has the Climate Change 
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Network in the Canadian Forest Service and 
the Program of Energy Research and 
Development in the Energy Sector. In addition, 
the MOU on S&T for sustainable development 
has facilitated joint activities. 


In our opinion, while these mechanisms go 
partway toward encouraging collaboration 
among departments and with external partners, 
in the absence of a comprehensive federal 
approach to the science of climate change 
based on common goals and clear expected 
results there could be gaps, duplication and 
wasted efforts in the research that is 
undertaken. To fulfil the spirit of the Strategy 
commitment on interdepartmental collaboration, 
we expected the government to adopt an 
approach to the science of climate change that 
would transcend individual departmental 
programs. In our view, some of the research 
efforts that are needed may not fit neatly into 
existing programs; all the necessary players 
may not be involved; and there is no clear 
mechanism for responding to new resource 
requirements. 


Joint performance review and reporting. 
Performance review and reporting are 
important principles of the Strategy. Every 
department and the federal government as a 
whole are accountable for the research results 
achieved and for reporting on those results. 


Performance review and reporting are essential 
for two reasons. First, they allow the 
government to adjust its goals and research 
plans in the light of new information and to 
deploy the resources required to achieve the 
goals. Second, they provide the basis for 
parliamentarians to assess whether the 
government's expenditures on science and 
technology reflect Canadian needs and 
opportunities, and to hold the government 
accountable for results. We expected that the 
government would develop means to review 
and report on the results of the federal 
research effort into climate change science 


either collectively, or separately in a way that 
would make clear the full extent of the federal 
effort. 


We found that climate change research is 
reported on a departmental basis, in order to 
align reporting with financial accountability. The 
existing departmental reporting mechanisms 
may limit the ability of the federal government 
to review and report on the joint federal efforts 
in climate change research, as well as 
Parliament's ability to undertake proper 
oversight. An alternative approach might 
include reporting fully in the Performance 
Report of a lead department, and in a summary 
way in the others’. 


We believe that better joint performance review 
and reporting as part of a government-wide 
process that involves stated goals and research 
plans and shared work would improve the 
management of climate change and 
accountability for its results. 


What departments are planning 


As part of meeting Canada’s Kyoto 
commitments to reduce greenhouse gas 
emissions, the Climate Change Action Fund 
was announced in the 1998 federal Budget. It 
has been allocated $150 million over three 
years, with an objective to leverage further 
funds from government and private sector 
organizations. It operates in four areas: 
foundation building, public education and 
outreach, science and adaptation, and 
technology. 
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The federal government is currently developing 
business plans for actions to be taken under the 
Climate Change Action Fund. We reviewed the 
proposed business plan for one of the four 
areas, science and adaptation, and found that it 
appears to address many of the weaknesses 
we have discussed in the government's 
approach to date. 


The business plan identifies common objectives 
and goals toward improving knowledge of 
climate change science. It builds on the 
planning activities that have taken place to date, 
such as the outputs from the MOU working 
group on climate change and variability, to 
identify gaps in the knowledge of both science 
and adaptation. It provides a general work plan 
that identifies the science and adaptation 
activities to be emphasized. A detailed work 
plan is being developed that will elaborate on 
specific deliverables, funding and timing. 


A key feature of the business plan is the 
proposed management framework to deliver 
the goals and work plans we have noted (see 
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the exhibit below), This framework will 
transcend departmental jurisdictions. 


In our view, the proposed structure 
demonstrates some of the key attributes of an 
effective management framework for a major 
initiative involving many stakeholders: 


e ministerial commitment; 


e an open, transparent process of agenda 
setting and developing work/action plans; 


e asupportive and funded secretariat to 
provide the necessary linkages across a 
complex set of structures; 


e adefined schedule of outputs, provided to 
ministers and against which performance 
can be measured; and 


a structure to respond to requirements for 
resource allocation or reallocation. 


Officials informed us that efforts are now under 
way to develop collective frameworks of 
evaluation and accountability to measure and 
report on results. 


Proposed Management Framework for Climate Change Science — 


Impacts and Adaptation Component 


Ministers 


Conclusion 


Our review of the approach taken thus far in the 
management of climate change science 
confirms that addressing horizontal S&T issues 
in a departmental management regime is an 
ongoing challenge. The mechanisms that are 
currently in place enable the participating 
departments to address research efforts within 
their existing departmental structures and 
programs. In our opinion, the management of 
climate change science to this point does not 
reflect fully the intent of the commitments made 
in the Strategy. 


The government has recognized that a more 
concerted approach is needed to optimize 
resources and co-ordinate research at the 
federal level and across all sectors, especially in 
view of its Kyoto commitments. In our opinion, 
the proposed business plan dealing with 
science, impacts and adaptation is a better 
reflection of the principles outlined in the 
Strategy. We believe that the proposed 
approach holds the promise of improved 
management of this horizontal issue. 


Deputy Ministers 


is 

a Liaison offices 
( 
A 


— Environment Canada (science) 


Federal Climate Change Secretariat 


— Natural Resources Canada (adaptation) 


Core Assistant 
Deputy Ministers 


Executive/Policy Committee 


Canadian Climate 
Program Board 


Technical Committee 


Source: Climate Change Action Fund, Draft Business Plan: Science, Impacts and Adaptation 
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(continued from page 22-13) 


Canada Foundation for Innovation and the 
Canadian Network for the Advancement 
of Research, Industry and Education 
(CANARIE Inc.). This may reflect a more 
general approach by government on how 
it sees it can best achieve its stated 
objectives. 


22.47 =‘ These and other spending 
decisions implicitly represent priorities of 
the government. However, in the Strategy, 
the government stated that it needed to 
conduct a more rigorous review of 
priorities. We would have expected that, 
at some point, the government would 
indicate the results of its rigorous review. 
This has not yet happened. The 
government needs to consider how best to 
communicate the results of this review of 
priorities to Parliament and other 
interested bodies. 


Intergovernmental co-operation and 
co-ordination have not taken the 
promised new direction 


22.48 The Strategy stated, ““An 
effective governance system for S&T in 
Canada demands closer co-operation and 
co-ordination of efforts among 
governments.” The government 
recognized that past high-level initiatives, 
although well-intentioned, had not 
resulted in much discernible progress. 
New approaches were needed to deal with 
specific issues like overlap and 
duplication and the sharing of 
information. 


22.49 The federal government’s 
commitments to intergovernmental 
co-operation and co-ordination are clearly 
listed in the Strategy. Where appropriate, 
new institutions and instruments were to 
be developed to facilitate improved 
intergovernmental relations, whether 
through regional or bilateral 
arrangements. To help shape this process, 
the government was to initiate 
results-oriented discussions with 
provincial and territorial governments. 


The Federal Science and Technology Strategy: A Review of Progress 


Those commitments established our 
expectations. Given the importance it 
placed on enhancing co-operation and 
co-ordination with other levels of 
government, we expected that the federal 
government would make this a high 
priority on its agenda. 


22.50 Since the spring of 1998, Industry 
Canada has been co-ordinating a network 
of federal-provincial officials to exchange 
information on science and technology. 
The network, led by Industry Canada, has 
completed an analysis of issues raised by 
provincial premiers and is preparing an 
inventory of initiatives taken on those 
issues. Efforts are now being made to 
develop an electronic channel of 
communication. Federal, provincial and 
territorial deputy ministers of industry met 
in September 1998 to discuss ways of 
improving the co-ordination of S&T 
activities. While we recognize that the 


' development of new institutions and 


instruments takes time, to date there is 
little evidence of tangible results. 


The government has started to report 
on its performance 


22.51 In 1996, we reiterated the need 
for consolidated information on the 
performance of departments and agencies 
on government-wide and horizontal 
issues, and on Canada’s performance 
overall in science and technology. In 
particular, this meant that 
government-wide performance 
expectations were needed, along with 
performance monitoring and reporting to 
Parliament. In December 1997, the 
government published its first annual 
report on S&T activities, Minding Our 
Future. That document states, “This 
report, the first in a series, provides an 
overview of the federal science and 
technology investment, reviews the 
federal government’s performance against 
the goals outlined in the Strategy, and 
identifies some challenges that must be 
overcome in the transition to a knowledge 
society.” 


The government needs 
to consider how best 
to communicate the 
results of its review of 


S&T priorities. 
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In December 1997, the 
government published 
its first annual report 
on S&T activities, 
Minding Our Future. 


We believe that the 
report constitutes a 
step in the right 
direction. However, we 
also found areas for 
significant 
improvement. 


22.52 We believe that the report 
constitutes a step in the right direction. 
However, we also found areas for 
significant improvement. The report needs 
to focus more on results. If the 
government wants to review its 
performance against the goals outlined in 
the Strategy, it needs to set meaningful 
performance expectations for each of 
them. Otherwise, it will be difficult to 
assess the extent to which the activities 
contribute to achieving the three goals of 
the Strategy. For example, the report 
highlights the various activities 
undertaken by departments that contribute 
to the goal “improved quality of life”, but 
does not indicate whether they are 
sufficient or what remains to be done. 


22.53 The report also provides little 
consolidated information on the 
performance of departments and agencies 
in horizontal areas. This kind of 
information is important to an 
understanding of spending priorities and 
as an input for discussions on 
accountability for priority setting. 
Although some progress has been made, 
we believe that parliamentarians do not 
have a firm basis for assessing whether the 
government’s expenditures on S&T reflect 
Canadian needs and opportunities, and for 
holding the government accountable for 
results. 


22.54 We expect that work undertaken 
by Statistics Canada will help to provide 
more information on the relationship 
between government involvement in 
science, technology and innovation and 
the three goals outlined in the Strategy. In 
1996, Industry Canada funded the 
Information System for Science and 
Technology Project, which is led by 
Statistics Canada. As part of this project, 
work has been undertaken to provide 
better information on where the 
government spends its S&T resources and, 
more important, what it gets in return. 
However, the latter issue will require that 
new indicators be developed and so, 


building on the work done thus far, 
Statistics Canada has initiated a 
Framework for a Statistical Information 
System — a tool to develop performance 
indicators for science and technology 
activities. The Framework will provide 
information on outcomes of S&T 
activities. Statistics Canada officials told 
us they were preparing a strategic plan to 
implement the Framework. At the end of 
our review, funding for this initiative had 
yet to be finalized. 


22.55 The annual report on S&T 
activities would be a good opportunity for 
the government to also lay out a 
forward-looking agenda. This could 
include the government’s views on the 
work of the ACST and the Council of 
Science and Technology Advisors, as well 
as results-oriented implementation plans. 


Implementation plans for the Strategy 
are still needed 


22.56 In our 1996 follow-up report we 
said that implementation plans for various 
initiatives planned under the Strategy had 
yet to be prepared. We noted the need for 
implementation plans that would include 
key steps, schedules, milestones and 
resource requirements and that would 
clearly establish accountability for results. 
We were informed at the time by Industry 
Canada that the Minister of Industry 
would “table a plan for the 
implementation of a large part of the 
Strategy in the autumn of 1996”. The 
Minister of Industry outlined 
implementation elements for the Strategy 
in October 1996. However, this outline 
was not supported by an appropriately 
detailed plan for carrying out the intended 
commitments. Our review suggests that 
there is still much to do, and that a plan 
for doing it would be beneficial. For 
example, the decision-making structure 
has been established, but how efforts will 
be co-ordinated needs to be planned and 
integrated with work on other outstanding 
commitments. 
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Progress in Departments Varies 


22.57 As well as making several 
commitments at the government-wide 
level, the Strategy stated that a common 
framework of operating principles was 
needed to ensure that departments and 
agencies acted together to reach S&T 
goals. The principles and associated 
departmental commitments were also 
supposed to provide a basis for measuring 
and evaluating results under the new 
management regime outlined in the 
Strategy. The seven operating principles 
are presented in Exhibit 22.2. 


22.58 Our follow-up on progress to date 
in implementing the operating principles 
covered a sample of four departments with 
significant science programs: Agriculture 
and Agri-Food Canada, Environment 
Canada, Fisheries and Oceans and Natural 
Resources Canada. Exhibit 22.3 provides 
a brief profile of the four departments and 
Exhibit 22.4 shows the financial resources 
they invest in science and technology 
activities. 


22.59 The Strategy stated that the way 
departments and agencies applied its 
principles to their S&T activities would 
vary, depending on their roles and 
responsibilities. We took this into account 
in our follow-up. We also had to consider 
that the departments did not all start from 
the same point. Some had already been 
applying a number of the principles 
outlined in the Strategy when it was 
announced. Therefore, while we expected 
each department to be working to apply 
the operating principles, we did not expect 
that they would all be at the same stage of 
progress. However, the experience of the 
most advanced can help the others to 
catch up. 


22.60 Although we found that progress 
in acting on the seven operating principles 
has varied considerably in the departments 
we examined, even the less advanced 
departments have moved in the direction 
desired by the government. Overall, 
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however, much more remains to be done, 
especially in reporting on performance. 


22.61 Our findings suggest that Natural 
Resources Canada is well on its way to 
putting the principles of the Strategy into 
practice. Environment Canada and 
Agriculture and Agri-Food Canada are 
following behind, having made progress in 
some areas but not in others. Fisheries and 
Oceans, with the smallest expenditure on 
S&T of the four departments, has made 
progress on several principles but has yet 
to make significant progress on others. 


22.62 Departments need to consider the 
merits of assessing their own progress to 
date on each of the commitments, and to 
set out plans for fully implementing the 
Strategy. 


22.63 As the departments continue to 
address the Strategy commitments, in our 
view there are three departmental science 


Although progress in 
acting on the seven 
operating principles 
has varied 
considerably, even the 
less advanced 
departments have 
moved in the desired 
direction. 


’ management issues that need special 


attention: mission-driven, results-based 
research; scientific excellence; and 
partnerships. 


Mission-driven, results-based research 


22.64 In 1994 we recommended that 
departments set clearer research goals and 
priorities; focus on achieving results; 
better identify both potential uses for, and 


Exhibit 22.2 


Common Framework of Operating Principles 


— Increasing the effectiveness of federally supported research 
— Capturing the benefits of partnerships 
— Emphasizing preventive approaches and sustainable development 


— Positioning Canada competitively within emerging international regulatory, 
standards and intellectual property regimes 


— Building information networks: the infrastructure of the knowledge economy 
— Extending science and technology linkages internationally 
— Promoting a stronger science culture 


Source: Science and Technology for the New Century — A Federal Strategy, 
Government of Canada, March 1996 
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users of, those results; and be held e prepare a report on its priorities, key 
accountable for making these changes. initiatives, spending plans, management 
The government accepted our challenges and performance measures for 
recommendations. S&T as part of what is now the annual 


Report on Plans and Priorities. 
22.65 The Strategy contained several 
specific commitments aimed atimproving 22.66 Our follow-up review found that 


information on departmental plans and none of the four departments had a 
performance. Each science-based distinct S&T plan. However, Environment 
department and agency was to: Canada and Natural Resources Canada 


have made good efforts to describe in their 
e set clear S&T targets and objectives, business plans how science and 
establish performance indicators based on technology activities contribute to the 
outputs and develop evaluation achievement of their policy objectives. 
frameworks; We also noted that some sectors of Natural 
Resources Canada and the Research 
e prepare an S&T plan describing and Branch at Agriculture and Agri-Food 
integrating the approach that it would take Canada have produced their own business 


within its business plan; and plans. 
Exhibit 22.3 Agriculture and Agri-Food Canada Environment Canada 
Science and Technology Agriculture and Agri-Food Canada’s mandate Environment Canada undertakes programs to 
Activities of Four Departments for research and development is “to improve reduce risk to human health and the 
the ongoing competitiveness of the Canadian environment. It provides weather and 
food and agriculture sector through the environmental predictions and warnings, as 
development and transfer of innovative well as emergency preparedness services to 
technologies”. S&T activities are conducted enhance safety from environmental hazards. 
through 18 research centres across the Over 80 percent of the Department’s 
country. The research centres study soil expenditures are on S&T activities and more 
properties, plant breeding, water use and than two thirds of its employees are classified 
management, energy, environmental quality, in S&T positions. 


production development including animal 
crossbreeding, feedlot systems, genetics, 
processing, distribution, retailing and 
consumer concerns. 


Fisheries and Oceans Natural Resources Canada 

Science in the Department involves the NRCan conducts leading-edge research in 
collection, analysis and interpretation of data science and technology in the area of forestry, 
in the fields of fisheries biology, aquaculture energy, metals and minerals and earth science 
science and oceanography, fish habitat and the in order to create and transfer the ideas, 
marine environment, and hydrography. Using knowledge and technologies that Canada 

this analysis and interpretation, science needs to use its resources wisely and 
provides timely advice in support of efficiently, to reduce costs, to protect the 
management for the conservation, protection, environment, and to help Canadians create 
and sustainable utilization of marine and new products and services. The science and 
aquatic resources, and for safe navigation. technology activities of NRCan are 


particularly important for meeting the 
challenge of sustainable development. The 
integration of economic, social and 
environmental considerations into resource 
decision-making requires a base of sound 
scientific knowledge. 


Source: Statistics Canada 
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22.67 We found considerable variation 
in the information on S&T activities 
presented in the four departments’ Reports 
on Plans and Priorities. More specifically, 
Agriculture and Agri-Food Canada and 
Fisheries and Oceans provide far less 
information than Natural Resources 
Canada and Environment Canada on S&T 
priorities and targets, management 
challenges and performance measures. For 
example, Fisheries and Oceans provides 
little information on its science priorities 
and describes only in broad terms the kind 
of science it wants to undertake. In 
contrast, for each of its policy goals 
Natural Resources Canada provides 
information on its related science and 


technology activities. 


22.68 Natural Resources Canada and 
Environment Canada have also taken a 
step forward, through S&T management 
frameworks, to improve the way they 
review, assess and report on their S&T 
activities. Natural Resources Canada’s 
efforts in this area are well under way; it 
started to implement its framework more 
than two years ago. Environment Canada 
recently published its own framework and 
is developing supporting documents. We 
encourage the other two departments to 


consider the same approach. 
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22.69 We found that progress in 
performance measurement has been slow 
except at Natural Resources Canada and, 
to a lesser extent, Environment Canada. 
As part of implementing their sustainable 
development strategies, most departments 
have developed a framework of 


performance measures. The next step, for Their S&T 

those departments that have not yet done management 

so, is to consider the benefits of building 

on this work by developing performance frameworks represent 
measures specifically for science a step by Natural 


activities. 
Resources Canada 


and Environment 
Canada toward 
improving the way 
they review, assess 
and report on their 
S&T activities. 


Scientific excellence 


22.70 The Strategy stated that the 
merits of a particular activity would be 
best confirmed through external review — 
an independent assessment of the design, 
performance and impact of a research 
effort. We found that all four departments 
have established or reorganized 


‘multidisciplinary advisory bodies. Most of 


them are expected to meet at least once or 
twice a year. Environment Canada has 
also created working groups drawn from 
its advisory committee members and 
employees to develop work plans for 
priority issues identified by the advisory 
committee. In our opinion, the next step 
would be to monitor management’s action 


Exhibit 22.4 


Science and Technology Spending 
on Research and Development and 
Related Scientific Activities by Four 

Departments in Selected Years 


Related Scientific Activities be 


Research and Development Fal 


1994-95 
1997-98 


Natural 


Canada Source: Statistics Canada 
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The Strategy asked 
departments and 
agencies to seek more 
opportunities for 
partnership 
arrangements as a way 
to increase 
effectiveness. 


on advisory committee recommendations. 
Some departments have already planned 
to do so. 


22.71 While expert advice is sought for 
research planning, expert review of the 
science quality and impact of individual 
projects appears to be less common. As 
we did not audit the process of external 
review in the departments, we cannot 
comment on its effectiveness. However, 
we believe this is an important issue and 
will address it in our future work. 


Partnerships 


22.72 All departments work with 
partners to carry out many of their 
activities. The Strategy asked departments 
and agencies to seek more opportunities 
for partnership arrangements as a way to 
increase effectiveness. The Strategy 
commits them to develop strategies for 
promoting partnerships and S&T 
collaborative arrangements with industry, 
the provinces, universities and other 
stakeholders. Departments were also 
asked to develop strategies for increasing 
interdepartmental collaboration to 
combine resources and eliminate 
duplication. 


22.73 ‘In our view, this process would 
translate into clearly stated objectives and 
expected results to be pursued through 
external arrangements. We expected that 
departments would be able to describe and 
report the flow of financial or other 
resources to them, and between them and 
the external participants in the 
arrangements. 


22.74 We found that progress has 
varied. Natural Resources Canada has 
made the most progress in setting out a 
strategy for partnership arrangements. In 
its public documents, it states specific 
objectives that it wants to achieve in 
partnership with all its stakeholders in 
order to meet each of its policy goals. 
Some of the sectors’ business plans also 
state in more detail what they intend to 
achieve with their partners. 


22.75 In 1997, Natural Resources 
Canada published a Framework for 
Revenue-Generation, External Funding 
and Collaborative Activities. The 
Framework is intended to, among other 
things, provide a set of common principles 
to guide decision making and 
implementation of new arrangements 
across the Department. Although the 
systems for gathering the information on 
its different revenue-generating activities 
are not yet fully in place, it appears that 
the Department will soon be in a position 
to report on its performance in these areas. 


22.76 We noted that Agriculture and 
Agri-Food Canada has created the 
Matching Investment Initiative, which has 
proved to be a successful way of 
increasing the level of its collaborative 
research with industry. From our review of 
documents and our discussions with 
officials of Agriculture and Agri-Food 
Canada, we found that although the 
Department clearly stresses the need to 
work with partners in pursuit of 
objectives, it makes few references to 
what needs to be done and with whom. 


22.77 Partnerships are an integral 
aspect of Environment Canada’s S&T 
programs. We found that it has made 
efforts to demonstrate how its partnerships 
will help it to meet its objectives. It has 
also identified areas where partnerships 
need to be built or strengthened. In neither 
case, however, did we find specific 
strategies in place. The Department 
informed us that management was to have 
discussed strategic issues related to S&T 
partnerships over the summer of 1998. 
The Department is also drafting a 
guideline document on S&T partnerships 
as part of its S&T Management 
Framework. In our opinion, the document 
needs to contain guidelines for reporting 
on the Department’s external partnership 
arrangements. 


22.78 Recognizing that it is not yet 
systematically looking for partnering 
opportunities, Fisheries and Oceans is 
developing a Partnering Strategy as part of 
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its Science 2005 Strategic Plan. It 
informed us that it has also begun to 
quantify the different types of external 
arrangements it has with its stakeholders. 


Conclusion 


22.79 As part of its S&T Strategy, the 
federal government committed itself to 
better managing its S&T activities. We 
looked at whether the government has 
made progress toward fulfilling its 
commitments to establish new institutions 
and mechanisms and to act on new 
operating principles for S&T, thereby 
placing the new management system on 
the solid foundation essential to it. 


22.80 Although the government has 
acted on some specific commitments, the 
establishment of a revised management 
system has been slow. Much remains to be 
done to make it work well. Since changes 
take time, we were looking for signs that 
results were “just around the corner’. We 
found that S&T issues are getting 
attention at a higher level. However, we 
could not conclude clearly whether all the 
actions taken so far were attributable to a 
new way of doing business based on new 
mechanisms or simply to the initial 
momentum of the Strategy. 


22.81 Whatever the explanation, the 
message remains the same: there are still 
significant challenges ahead because the 
government has not yet acted fully on its 
commitments under the Strategy. Two 
comments are warranted. First, it may be 
time for the government to clearly set out 
what remains to be done — what is next 
on the agenda. This would send a strong 
signal that the Strategy is alive and well 
and has its second wind. 


22.82 Second, as we stated in 1994, 
leadership that transcends departmental 
mandates is essential to meet these 
challenges, especially if the progress to 
date has been due more to the initial 
momentum than to enduring change. In 
our opinion, the leadership shown so far in 
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co-ordinating the S&T policy will not be 
sufficient in the future to address the 
remaining challenges. In fact, leadership 
will be a critical factor in ensuring that the 
current Strategy does not become simply 
one more missed opportunity. 


22.83 In departments we found that 
although their progress in acting on the 
Strategy’s seven operating principles has 
varied considerably, even the less 
advanced have moved in the direction 
desired by the government. Overall, 
however, much more remains to be done, 
especially in reporting on performance. 


22.84 The extent to which individual 
departments are improving the 
management of their science activities 
becomes less relevant, however, if they 
are not working together to better manage 
horizontal issues. Although we observed 
increased co-operation among 


‘departments, our case study on the 


horizontal issue of climate change science 
demonstrates that, in this particular case, 
departments are still carrying out science 
activities on the basis of their own 
priorities rather than common goals. 


22.85 This has started to change, 
however. As part of the case study, we 
found that the proposed business plan for 
science activities under the Climate 
Change Action Fund appears to address 
the weaknesses we have identified in the 
government’s current approach to S&T in 
general. This and other recent initiatives 
lead us to believe that the experience in 
this area could lay a new foundation for 
the management of other horizontal 
issues. 


Joint response of Agriculture and 
Agri-Food Canada, Environment 
Canada, Fisheries and Oceans, Industry 
Canada and Natural Resources Canada: 
A healthy federal science and technology 
community is essential to Canada’s 
economic, social and environmental 
well-being. The Federal Science and 
Technology (S&T) Strategy, delivered 
through each department’s mandate, is a 


Leadership will be a 
Critical factor in 
ensuring that the 
current Strategy does 
not become simply 
one more missed 
opportunity. 


It may be time for the 

government to clearly 
set out what remains 

to be done — what is 

next on the agenda. 
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blueprint for the direction and 
management of federal science and 
technology. We were pleased to note the 
Auditor General’s recognition of the fact 
that the application of the Strategy’s 
operating principles across departments 
varies according to their mandates. We 
welcome this follow-up as an indication of 
the priority that government places on the 
effectiveness of its S&T resources. While 
the data presented in the chapter are 
accurate, this response presents an 
opportunity to provide a context for the 
information as well as to highlight areas 
of continued progress since the audit was 
completed (note: this response speaks only 
for the five departments included in this 
follow-up.) 


Departments have been working hard 
within their respective accountabilities 
and mandates to implement the Strategy, 
although the progress has been affected 
significantly by Program Review. In 
accomplishing this, and in support of the 
government’s overall policy objectives, we 
welcome the support of the Minister of 
Industry in helping to achieve horizontal 
policy co-ordination on key issues. We feel 
it is essential to recognize that different 
parts of the federal S&T effort (e.g. 
advancing the frontiers of knowledge and 
mission-driven research) require different 
planning approaches. While the Strategy 
recognizes that federal S&T ministers 
must remain accountable for directing 
their departments’ S&T efforts and 
resources toward fulfilling their mandates, 


we agree that some key files (e.g. climate 
change) do benefit from a horizontal 
management approach. 


Our implementation of the Strategy 
continues and includes improving 
management of S&T human resources and 
building effective partnerships among 
levels of government and non-government 
stakeholders. Partnerships and 
collaborations include the 
interdepartmental S&T human resource 
initiative, the Climate Change Action 
Fund and the MOU on Science for 
Sustainable Development. The MOU itself 
has had a major impact in strengthening 
working relationships and increasing 
collaboration on key issues, such as 
climate change science, metals in the 
environment, nutrients and endocrine 
modulating substances. The success of the 
MOU will best be measured in the success 
of its individual projects, most of which 
are still in progress. The MOU was 
renewed in 1998 and was expanded to 
include Health Canada. The results of 
such co-operation and collaboration in 
S&T are becoming more evident as 
experience with these mechanisms and 
approaches grows. There are several other 
areas, such as Northern S&T and toxics 
substances research, where we are 
collectively just beginning to apply this 
experience. We feel that our efforts are 
having positive impacts and will continue 
to influence the co-ordination and 
co-operative management of federal S&T 
in the future. 
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ty About the Follow-up 


Objective 


Science and Technology for the New Century — A Federal Strategy is a policy statement that provides 
direction to departments and sets out the elements of a federal governance system for science and technology. 
Our objective in reviewing the Strategy was to assess the extent of progress made by the government in 
establishing new institutions and mechanisms for governance, and by departments in applying the operating 
principles set out in the Strategy. | 


Scope 


We brought forward issues raised in our September 1996 Report Chapter 15 — Federal Science and 
Technology Activities: Follow-up. 


We reviewed the commitments under the Strategy that are aimed at establishing new institutions and 
mechanisms for governance. 


The Strategy outlines a framework of operating principles that provides direction to departments on how to 
manage their respective science programs. We reviewed the progress of selected departments (Agriculture and 
Agri-Food Canada, Environment Canada, Fisheries and Oceans and Natural Resources Canada) in applying 
these principles. 


We prepared a case study whose purpose was to review whether the management of climate change science 
activities in the four selected departments reflects the principal commitments of the Strategy. 


Approach 


As called for in the Strategy, all major federal departments and agencies engaged in S&T prepared action 
plans detailing how they would put the Strategy into effect. We reviewed those action plans. 


The federal government published a report titled Minding Our Future, A Report on Federal Science and 
Technology — 1997. Each department produced a summary report on its activities under the Strategy as input 
to Minding Our Future. We reviewed those summary reports. 


We held interviews and discussions with officials involved in science and technology and reviewed relevant 
documentation. 


Audit Team 


Assistant Auditor General: Richard Flageole 
Principal: Peter Simeoni 


Ian Campbell 
Genevieve Hivon 
Sandra Lopes 
Michelle Salvail 
Marilyn Taylor 


For information, please contact Peter Simeoni. 
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Chapter 23 


Veterans Affairs Canada 


Disability Pensions 


The audit work reported in this chapter was conducted in accordance with the legislative 
mandate, policies and practices of the Office of the Auditor General. These policies and 
practices embrace the standards recommended by the Public Sector Accounting and 
Auditing Board (PSAAB) of the Canadian Institute of Chartered Accountants. The 
numbered paragraphs in bold face represent recommendations. 
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Veterans Affairs Canada 


Disability Pensions 


Main Points 


23.1 The disability pension program has annual expenditures of about $1.1 billion, making it the largest 
program of Veterans Affairs Canada. Disability pensions are paid to approximately 151,500 recipients — 68,000 
veterans, 25,500 former or current peacetime members of the Canadian Forces and 58,000 survivors. 


23.2 In September 1995, pension reform legislation came into effect. The legislation resulted in fundamental 
changes in responsibilities for the disability pension program. The Department took on new roles in providing 
assistance and counselling to disability pension applicants and in adjudicating first decisions for pension claims. 
The Veterans Review and Appeal Board was formed at that time to hear reviews and appeals of decisions. 


23.3 The main focus of the disability pension program since pension reform has been on reducing the time it 
takes to make pension decisions. Veterans Affairs Canada and the Veterans Review and Appeal Board have 
exceeded their targets for reducing turnaround times for disability pension decisions. 


23.4 The quality and consistency of the services provided in the preparation of first applications could be 
improved through greater use of standardized guidance materials and training for all those involved in the 
preparation of first applications. The table of disabilities, which is used in determining the nature and extent of 
disability for pension purposes, provides limited guidance for assessing the extent of some disabilities. To ensure 
consistent and high-quality disability pension decisions, the Department needs to improve its guidance materials 
that facilitate entitlement and assessment adjudications. 


23.5 Veterans Affairs Canada does not provide reasons for assessment decisions. There is a need for the 
Department and the Veterans Review and Appeal Board to regularly analyze the reasons for overturned pension 
decisions. 


23.6 Veterans Affairs Canada is not fully utilizing the departmental review process. This process was put in 
place by pension reform legislation and allows certain initial decisions to be reviewed more efficiently and in less 
time than a formal review request to the Veterans Review and Appeal Board. 


23.17 Within the next 10 years, the mix of disability pensioners will change considerably, with a significant 
increase in the number of former or current peacetime members of the Canadian Forces. The Department needs to 
inform Parliament of the changes in its client base and the strategy for addressing those changes. 


23.8 Veterans Affairs Canada has assigned a high priority to its Year 2000 projects. It has a plan and schedule 
that is expected to achieve Year 2000 compliance for its mission-critical systems by the end of 1998. However, the 
Department is also dependent on other systems for information, and disruption of those systems could affect the 
delivery of the disability pension program. 
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Introduction 


Background and purpose of the 
disability pension program 


23.9 In 1997-98, Veterans Affairs 
Canada spent about $1.9 billion to provide 
veterans, qualified civilians and their 
families with the benefits, financial 
assistance and health care services to 
which they are entitled. Approximately 
$1.1 billion of the total amount was spent 
on providing disability pensions to war 
veterans, members and former members 
of the Canadian Forces and survivors. The 
objective of the disability pension 
program is “to compensate veterans and 
other eligible clients in a timely and 
equitable manner for death or disabilities 
related to military service.” 


23.10 Canada first began paying 
veterans pensions in 1917. After the 
Second World War, the veterans charter 
was established, giving veterans certain 
rights, privileges and benefits. The 
primary obligations undertaken by the 
government were the care and 
compensation of veterans who returned 
home wounded and disabled and the 
re-establishment of returning veterans into 
civilian life. The provision of disability 
pensions continues to be the largest single 
component of these ongoing obligations. 
In addition, the disability pension program 
is a gateway that provides eligibility to 
other programs provided by the 
Department, particularly health care 
benefits. 


Responsibility for pension decisions 


23.11 The Veterans Affairs Portfolio 
comprises Veterans Affairs Canada and 
the Veterans Review and Appeal Board. 


23.12 The Department is responsible 
for counselling and assisting pension 
applicants, making the initial decisions on 
disability pensions, determining the 
amount of pension awards and 
administering the payment of awards. As 


eee ee 
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well, the Department’s Bureau of Pension 
Advocates provides advocates to represent 
veterans and other eligible clients 
appearing before the Veterans Review and 
Appeal Board. 


23.13 The Veterans Review and Appeal 
Board is an independent body reporting to 
Parliament through the Minister of 
Veterans Affairs. Board members are 
Governor in Council appointees. The 
Board is responsible for making decisions 
on reviews and appeals by applicants who 
are dissatisfied with pension decisions. 


23.14 In 1997-98, the cost of delivering 
disability pension benefits was 
approximately $29 million for the 
Department (including $5 million for the 
Bureau of Pension Advocates) and 

$8 million for the Veterans Review and 


Appeal Board. Approximately 


$1.1 billion is spent 
annually on disability 
pensions. 


The pension recipients 


23.15 Veterans Affairs Canada 
administers disability pensions under the 
Pension Act to veterans of the First and 
Second World Wars and members of the 
Canadian Forces who served during 
peacetime, under the Veterans Benefit Act 
to veterans of the Korean conflict, and 
under the Merchant Navy Veteran and 
Civilian War-related Benefits Act to 
Merchant Navy veterans and qualified 
civilians. The Department also 
administers disability pensions to disabled 
peacekeepers as designated by the Special 
Duty Area Pension Order. In addition, it 
provides adjudication and counselling 
services, but does not pay pensions, on 
behalf of the Royal Canadian Mounted 
Police. 


23.16 There are three major groups 
within the population of disability 
pensioners. The first group consists of 
veterans of the First and Second World 
Wars and the Korean conflict. This group 
of war service veterans has an average age 
approaching 78 years. 


23.17 The second group of pension 
recipients consists of those who served in 
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Forty-nine percent of 
new Clients were 
members of the 
Canadian Forces. 


Exhibit 23.1 


Disability Pension 
Program Clients 


Actual 1995 to 1998 
Forecast 1999 to 2007 


War Service Veterans 
Canadian Forces - - - - - 


Survivors: —=—— 


Source: Veterans Affairs 
Canada — Corporate 
Planning Division 


the Canadian Forces during peacetime, 
including those who served in Special 
Duty Areas. Special Duty Areas are 
usually associated with United Nations 
peacekeeping missions and are listed in 
Chapter 350 of the Consolidated 
Regulations of Canada. Peacetime 
pension recipients are generally younger 
than war veterans; most are under 65 years 
old. In conjunction with National 
Defence, initiatives are currently under 
way to improve service to these clients 
through better working arrangements and 
information exchange between the two 
departments. These initiatives are 
discussed further in paragraphs 23.78 and 
23eto, 


23.18 The third group of pension 
recipients includes survivors of 
pensioners. Survivor benefits are based on 
the pension that had been or would have 
been awarded to the pensioner prior to 
death. 


23.19 Composition of clientele is 
changing. The total number of recipients 
of disability pension benefits has remained 
relatively constant over the past decade. In 
1997-98, the Department provided 
pension benefits to a total of 151,500 
recipients, consisting of 68,000 veterans, 


Number of Clients 
(thousands) 


Th 


25,500 current or former peacetime 
members of the Canadian Forces and 
58,000 survivors. The composition of the 
clientele is shifting from war service 
disability pensioners toward survivors and 
peacetime disability pensioners. As shown 
in Exhibit 23.1, the Department estimates 
that by the year 2007, its disability 
pension clients will total about 145,000. 
Although this number is comparable with 
the current number, the mix of recipients 
will be significantly different, consisting 
of about 38,000 veterans, 41,000 former 
or current members of the Canadian 
Forces and 66,000 survivors. 


23.20 In 1997-98, approximately 49 
percent of new disability pension program 
clients were members or former members 
of the Canadian Forces who served in 
peacetime or personnel who served in a 
Special Duty Area. By the year 2006-07, 
the Department projects that this figure 
will rise to 60 percent. The number of 
applications for disability pension benefits 
received from this client group is a 
significant factor in the future of the 
Department, as the average age of these 
applicants is under 65 years. Exhibit 23.2 
compares the age distribution of war 
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service veterans and Canadian Forces 
disability pensioners. 


Entitlement to disability benefits is 
determined by nature of service 


23.21. The Pension Act incorporates two 
bases of entitlement to disability pension 
awards — the insurance principle and the 
compensation principle. 


23.22 The insurance principle applies to 
clients with wartime service and includes 
peacekeepers who served in Special Duty 
Areas such as Haiti, Rwanda or the former 
Yugoslavia. Entitlement is granted on the 
basis of whether disability or death is 
attributable to, aggravated by or incurred 
during service. The government assumes 
complete responsibility for death, illness 
or injury 24 hours a day. Members of the 
Canadian Forces who qualify under the 
insurance principle, such as injured 
peacekeepers, can receive compensation 
while serving in the Canadian Forces. 


23.23 For peacetime members of the 
Canadian Forces, entitlement to disability 


Number of Disability Pensioners 
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pension benefits is compensation for 
disabilities that are service-related. 
Currently serving members of the 
Canadian Forces may submit applications 
for disability pensions to Veterans Affairs 
Canada. For those eligible under the 
compensation principle, benefits 
commence upon discharge from the 
Canadian Forces. Benefits commence 
immediately for members of the Reserve 
Force who receive a favourable decision 
under the compensation principle. 
Exhibit 23.3 provides a summary 
comparison of the applicability of the 
insurance and the compensation 
principles. 


23.24 Disability pensions are awarded 
based on the extent to which a disability is 
related to service (entitlement) and on the 
nature and extent of the disability 
(assessment). Favourable entitlements for 
pension awards are rendered in fifths. 
Awards of one-fifth to four-fifths 
entitlement indicate that the disability is 
deemed to be partially service-related. 


Exhibit 23.2 


Age Distribution of Disability 


Pensioners, 31 March 1998 


In 1997-98, 1,767 pension 


program clients were in 
receipt of non-disability-based 
awards such as 


prisoner-of-war compensation 
or compassionate awards. 


These clients are not included 
in the age distributions. 


War Service Veterans 


Canadian Forces - ------ 
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Source: Veterans Affairs Canada 
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Exhibit 23.3 


23.25 The pension legislation includes 
a long-standing “benefit of the doubt” 
provision. This provision requires, in 
making a decision, that every reasonable 
inference in favour of the applicant be 
drawn, that any uncontradicted evidence 
presented by the applicant that is credible 
in the circumstances be accepted, and that 
the decision resolve in favour of the 
applicant any reasonable doubt as to 
whether the applicant has established a 
case. 


Assessing the extent of disability for 
pension awards 


23.26 The basic concept established by 
the Pension Act in 1919 was that 
assessment of the disability of a pensioner 
was to be based on the extent to which the 
capacity for earning a living in the general 
labour market had been lessened. Today, 
under the Pension Act, “disability” means 
the loss or lessening of power to will and 
to do any normal mental or physical act. 


23.27. A diagnosis and medical 
evidence are required to assess the nature 
and extent of disability for pension 
purposes. The Pension Act establishes the 
requirement for a table of disabilities to 
provide guidance to physicians and 
surgeons in assessing the extent of a 
disability. 


Comparison of the Insurance and Compensation Principles in the Pension Act 


Determining pension amounts 


23.28 The determination of entitlement 
in fifths and the assessment of the extent 
of disability by percentage are combined 
to arrive at a disability percentage for the 
purpose of awarding a pension. 

Exhibit 23.4 shows the distribution of war 
service and Canadian Forces disability 
pensioners by percentage of disability. 


23.29 Monthly pension rates are based 
on the extent of disability, marital status 
and number of eligible dependants. These 
benefits are not subject to personal income 
taxes and are not income-dependent. 
Exhibit 23.5 provides examples of the 
1998 monthly disability pension rates. 
Survivor benefits are paid in proportion to 
the benefits that were being paid, or that 
would have been paid, to the disability 
pensioner at the time of death. One year 
following the date of death of the 
pensioner, disability pensions assessed at 
less than 48 percent are reduced by one 
half, while those assessed at 48 percent or 
more are converted to full survivors’ 
pensions, which equates to 75 percent of 
the basic single pension. 


Focus of the audit 


23.30 Our audit focussed on the 
delivery of disability pension benefits by 
the Veterans Affairs Portfolio. 


Pension Act 


Insurance Principle 


Compensation Principle 
Subsection 21(2) 


| Covers those who served in: 


Subsection 21(1) 

1. Theatre of War 
— World War I 
— World War II 


2. Theatre of Operations 
— Korean Conflict 


3. Special Duty Areas 


1. Canadian Forces 


Pension entitlement: 


Conditions incurred during, 


| of service 


aggravated by or attributable to time 


Conditions that arose out of, were 
aggravated by or directly connected 
with military service during peacetime 


Source: Pension Act, Part III — Pensions, subsections 21(1) and 21(2) 
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We examined the adequacy of the 
management practices followed to ensure 
that high-quality pension decisions are 
made on a consistent basis and in a timely 
manner. We also examined the 
Department’s progress in making its 
pension and allowance systems Year 
2000-compliant. Details on the objectives, 
scope and criteria of this audit are found 
at the end of the chapter in the section 
About the Audit. 


Observations and 
Recommendations 


Turnaround Times for Processing 
Pensions 


Pension reform presented challenges 


23.31 For many years, veterans and 
veterans’ organizations expressed concern 
and frustration about the disability pension 
process. The process has been the subject 
of numerous studies, both inside and 
outside the Veterans Affairs Portfolio. 
Prior to September 1995, four separate 


Number of Clients 
(thousands) 
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entities were involved in the pension 
process: Veterans Affairs Canada, the 
Bureau of Pensions Advocates, the 
Canadian Pension Commission and the 
Veterans Appeal Board. Each entity was 
governed by its own legislation and was 
responsible for performing specific tasks 
within the process. The system was 
complex, with many players handling 
each claim. There were excessively long 
application-processing times, a large 
backlog of claims, a high incidence of 
appeals and a high percentage of decisions 
being overturned on appeal. 


23.32 In September 1995, new and 
amended legislation was adopted to 
overhaul the application and appeals 
process for disability pensions. As part of 
this initiative, the government made a 
commitment to reduce turnaround times 
for processing pensions by one half, over a 
two-year period, without affecting 
veterans’ pension benefits or their right to 
appeal. The legislative amendments, also 
referred to as pension reform, resulted in 
the following changes: 


e The authority to make first decisions 
on claims for disability pension benefits, 


Exhibit 23.4 


Distribution of Disability 
Pensioners by Percentage of 


10 Total = 91,752 


Disability, 31 March 1998 


In 1997-98, 1,767 pension 
program Clients were in receipt of 
non-disability-based awards such 


as prisoner-of-war compensation 
or compassionate awards. These 
clients are not included in the 


percentage of disability 
distributions. 


War Service Veterans 
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Pension reform 
presented significant 


previously the responsibility of the 
Canadian Pension Commission, was 
transferred to the Department. 


e Responsibility for preparing first 
applications, previously a function of the 
Bureau of Pensions Advocates, was given 
to departmental officials in district offices. 


e The Bureau of Pensions Advocates, 
previously a separate agency responsible 
for both the preparation of first 


district office or service centre personnel 
to applicants in preparing a complete 
disability pension application supported 
by service records, physician statements 
and medical evidence. 


e Completed applications are 
submitted to the Department’s head office, 
where adjudicators render entitlement and 
assessment decisions. Departmental 
medical advisors are available for 
consultation on entitlement decisions and, 


challenges. applications and advocacy services for except for hearing loss claims, are 
appeals, was merged with the Department —_ consulted for all assessment decisions. 
and retained only the responsibility for Pact 
providing advocacy services for reviews c if id licants are not satisfied with 
and appeals. the first decision, they may request a 
departmental review upon presentation of 
e The Canadian Pension Commission, new evidence or they may request the 
which was previously responsible for first _ Veterans Review and Appeal Board to 
decisions and the first level of appeal, and —_— review the decision. 
the Veterans Appeal Board, previousl 
PP P y 23.34 Pension reform resulted in 
responsible for the second level of appeal, Beads Cars 
: ; significant challenges for all organizations 
were combined into the Veterans Review 
: involved in the pension process. For 
and Appeal Board to hear first-level and : 
example, Veterans Affairs Canada faced 
second-level appeals. ’ 
the challenge of meeting the targeted 
23.33 Asaresult of pensionreform, the eduction in turnaround times, while 
following procedures became the key assuming its new role of assisting in the 
steps of the pension decision-making preparation of disability pension 
process: applications, making first decisions on 
disability pension claims and integrating 
e Most applications are submitted the Bureau of Pension Advocates into the 
through one of forty district offices or Department. The Veterans Review and 
service centres across Canada. Assistance Appeal Board was formed to fulfil the 
and counselling services are provided by review and appeal roles previously 
Exhibit 23.5 
Examples of Monthly Disability Ree 
Pension Rates - Effective vb he oped 
1 January 1998 5% 15% 25% 35% 75% 100% 
Single Pensioner $85.81 $257.43 $429.05 $600.67 $1,287.15 $1,716.20 
Married Pensioner 107.26 321819 536.31 750.84 1,608.94 2,145.25 
Pensioner, Spouse and Two Children 126.57 379.71 632.85 885.99 1,898.55 2,531.40 
Proportionate Rates 
5% 15% 25% | 35% 50 - 100% 
rnp loco me Surviving Spouse $53.63 $160.89 $268.16 | $375.42 $1,287.15 
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provided by the Canadian Pension 


Commission and the Veterans Appeal 


Board. 


September 1997 turnaround time 


targets have been exceeded 


23.35 Turnaround time has been the 


key performance indicator for measuring 
the progress of the Veterans Affairs 
Portfolio in achieving its pension reform 
commitments. For this purpose, 


turnaround time is defined as the time 
from the date an applicant indicates an 


intention to apply for a pension, or appeal 
a previous decision, to the date the award 
is processed for payment. Unfavourable 
decisions are not included in the 


turnaround times. In Chapter 12 of our 


May 1996 Report to Parliament, we 
reported on our examination of the first 
application turnaround times reported by 
the Department prior to pension reform. 


We also reviewed the turnaround times 


reported for reviews and appeals. 


23.36 The Veterans Affairs Portfolio 


has achieved significant reductions in 


turnaround times for processing pensions, 


First Applications 


Reviews Appeals 
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and has exceeded its targets for September 
1997. The reduction in turnaround times 
has been achieved through a simplified 
organizational structure that resulted in 
fewer steps to reach decisions. The most 
significant reductions have occurred in the 
time for preparing first applications and 
for preparing cases for review and appeal 


hearings. Turnaround time 
23.37 We verified turnaround times targets have been 
reported for September 1997. Our 

examination was based on a detailed exceeded. 


review and compilation of turnaround 
times for a statistical sample of claims 
adjudicated during the period July to 
December 1997. From this examination, 
we determined that the information 
contained in the Pension Status and 
Inquiry System is accurate for the 
purposes of compiling turnaround time 
statistics. Exhibit 23.6 shows significant 
improvements in turnaround times for 
each type of claim. 


23.38 The turnaround times reported by 
the Department are average turnaround 
times for each claim type. The distribution 
of turnaround times for first applications 
before and after pension reform are 


Exhibit 23.6 


Turnaround Times for 
Processing Pensions 


Pre-Reform 1994-95 pe 
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Source: Veterans Affairs Canada 
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The Department 
receives 
approximately 14,000 
applications per year. 


Exhibit 23.7 


Distribution of Turnaround Times 
for Favourable First Applications 
- Pre- vs. Post-Pension Reform 


compared in Exhibit 23.7. As shown, the 
average turnaround time has decreased 
from 18 months to 5.4 months and the 
variability in turnaround times has 
decreased significantly. In other words, 
since pension reform, average turnaround 
time as a performance indicator has 
become more reliable because the actual 
turnaround times are more closely 
grouped around the average. Pension 
claims are processed in less time and in a 
more predictable amount of time. Such 
information is useful for operational 
planning, accountability and development 
of service standards. 


Service standards 


23.39 In June 1998, the Veterans Affairs 
Portfolio published a client information 
brochure entitled Veterans Affairs - At 
Your Service, outlining the quality of 
service that can be expected by clients. 
Such commitments to service are 
important elements of an accountability 
framework. The Department has 
established a service standard of 18 weeks 


% of Total Favourable First Applications 


(4.1 months) for reaching a decision once 
a completed first application has been 
received. 


23.40 During 1997-98, the Department 
received approximately 14,000 first 
applications for pension benefits. A first 
application is the first time that an 
applicant applies to receive a disability 
pension for a medical condition. It does 
not include requests to reassess existing 
pensioned conditions. 


23.41 The Department’s role in the 
pension application process includes 
providing assistance and counselling to 
applicants in preparing applications, 
obtaining information in support of the 
application, requesting and reviewing 
service records and obtaining current 
medical information. Exhibit 23.8 shows 
the distribution of turnaround times for the 
period January to June 1998. These 
turnaround times include all activities 
from the time an application is initiated to 
the time payment is requested. We noted 
during our audit that the process for 
preparing applications represented 


25 
Post-Pension Reform 
Average Turnaround Time = 5.4 months 
20 (based on a total of 2,883 first 
4 applications processed into payment 
: : during the period July to December 1997) 
‘ 
15 A 2 
' ‘ 
' . 
' N Pre-Pension Reform 
; ‘ Average Turnaround Time = 18 months 
10 ! ws (based on a total of 7,323 first 
' . . . 
; applications processed into payment 
' \ during the fiscal years 1992-93 
to 1994-95) 


Turnaround Time for Favourable First Applications 


Source: Pension Status and Inquiry System 
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approximately one half of the turnaround 
time. Consequently, given the significant 
amount of time to prepare an application, 
and the role played by the Department in 
this step, we would expect that this part of 
the process would also be covered by a 
service standard. 


23.42 Veterans Affairs Canada should 
develop service standards for the 
Department’s role in preparing a first 
application for disability pension 
benefits. 


Department’s response: The Department 
agrees and therefore included such 
standards in its 1998 brochure “Veterans 
Affairs At Your Service” in the sections 
“Meeting Your Needs” and “Information 
and Advice”. More specifically, in the 
section on “Disability Pension 
Application”, we also state: “If you 
contact us about a service-related 
disability pension, we will help you 
prepare your application, help you obtain 
information in support of your 
application, answer your questions, 
research your records and assist you to 


Favourable First 
Applications 
4.9 
800— 
Seep 
700 — 
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obtain current medical information.” We 
intend to survey our clients on the quality 
and effectiveness of these services. 


Veterans Affairs Canada chose not to 
include service standards about the time 
to carry out these services because most of 
the activities involved in case preparation 
lie outside the Department's control, 
either with outside agencies or clients 
themselves. The service standard brochure 
was the subject of consultation with the 
Department's clients and client groups 
who agreed that time standards for case 
preparation were of lesser significance 
than those we chose to include on decision 
making and medical exams. 


Making Pension Decisions 


Preparing the disability pension 
application 


23.43 The role of the pension officers is 
to provide counselling services to 
applicants and to assist them in preparing 
applications for disability pension 
benefits. The quality and completeness of 
the application is important in obtaining 


Exhibit 23.8 
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There are 
opportunities to 
improve the 
consistency and 


quality of services. 


the right pension decision the first time. 
High-quality first applications include 
complete and accurate information for 
each medical condition claimed, and are 
supported by appropriate service 
documents, medical records and 
physicians’ statements. 


23.44 The Department has developed a 
standard set of application forms for 
disability pension benefits. When 
completed, the application package should 
provide details of each medical condition 
claimed and its claimed relationship to the 
applicant’s service. 


23.45 We found inconsistencies in the 
level and nature of the services provided 
by the Department in counselling 
applicants. For example, in some offices, 
we noted that counselling was provided to 
applicants in identifying service-related 
disabilities other than those initially put 
forth by the applicants. In other instances, 
we found that the focus was limited to the 
conditions identified by the applicants. 


23.46 The Department provided 
national and regional training throughout 
the pension reform process. However, the 
Department does not have a formal 
training policy for the disability pension 
program and we noted that some persion 
officers have not yet received formal 
training. 


23.47 We also noted that the working 
tools used to assist in the preparation of 
first applications, including medical 
guidance materials and questionnaires, 
vary from office to office. 


23.48 We concluded that there are 
opportunities to improve the consistency 
and quality of the services provided to 
applicants in the preparation of disability 
pension applications. By improving and 
implementing standardized policies, 
practices and procedures, the Department 
can obtain greater assurance that disability 
pension applications are completely and 
accurately prepared and include the 
appropriate supporting evidence. 


23.49 The number of applications 
completed by each pension officer varies 
widely, depending on the nature of the 
claims. For example, hearing loss claims 
are more straightforward to process, 
compared with claims involving complex 
or multiple conditions, such as orthopedic 
injuries, which can be very difficult and 
time-consuming. In addition, the mix of 
clientele can be quite different from one 
district to another. For instance, 
nationally, the Canadian Forces members 
represented 45 percent of first applications 


- in 1997-98. However, in some district 


offices, Canadians Forces members 
accounted for up to 70 percent of first 
applications. The service documents for 
Canadian Forces members tend to be quite 
extensive, requiring more time to review 
than a typical war service veteran’s file. 
To ensure that the needs of claimants are 
met in an efficient and timely manner, the 
Department needs to review its 
distribution of work related to the 
preparation of first applications to 
determine where resources should be 
allocated. 


23.50 Veterans Affairs Canada should 
improve the services provided to 
applicants in the preparation of 
disability pension applications by: 


e improving standard national 
guidance materials and manuals; 


e ensuring that new pension officers 
receive appropriate initial training on a 
timely basis; and 


e re-examining workload 
distribution. 


Department’s response: The Department 
is committed to the pursuit of the highest 
standards of counselling and assistance to 
applicants in the preparation of disability 
pension applications and to the provision 
of timely and effective training to pension 
officers. 


Work is under way to develop both a 
standardized information and procedures 
manual for use by field staff and a 
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training module for pension officers to 
support the first application process. 


The Department agrees that workload 
distribution is an important consideration 
for effective resource utilization and will 
continue to actively address this issue. 
This will be done, in part, through its 
ongoing Workload Standards project and 
through medium- and long-term human 
resource planning based on employee 
demographic and client forecasts. 


Adjudication of first applications 


23.51 ‘First decision process. 
Adjudicators at Head Office in 
Charlottetown, who have health sciences 
or nursing backgrounds, make first 
decisions on disability pension claims. 
Departmental medical advisors are 
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available for consultation on issues 
relating to entitlement and, except for 
hearing loss claims, are consulted for all 
assessment decisions. The first decision 
process includes a determination of an 
applicant’s entitlement and an assessment 
of the extent of disability for pension 
purposes. 


23.52 ‘As illustrated in Exhibit 23.9, 
three key questions are addressed in the 
first decision process: 


e Is there a medical disability? 
e Does the disability relate to service? 
e What is the extent of the disability? 


23.53 As part of the first application, 
the applicant must specify the medical 
condition that is the basis for the claim 
and submit medical evidence to support 


Exhibit 23.9 
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of First Applications 
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the application. The adjudicator uses this 
information to determine if there is a 
medical disability. 


23.54 The application must also be 
supported by service records. The 
adjudicator uses this information to 
determine if the disability relates to 
service. This is the entitlement decision. 
For certain medical conditions and 
service-related activities, the Department 
does have guidance materials to assist 
adjudicators in making entitlement 
decisions. General guidelines have been 
developed to assist in determining 
entitlement for certain common claims, 
including hearing loss and orthopedic 
conditions; however, for most other 
conditions, the guidance to link medical 
conditions to service needs to be 
improved. 


23.55 Table of disabilities. The 
primary decision-making guidance 
material for assessing the extent of 
medical disability for pension purposes is 
the table of disabilities. Other guidance in 
the form of memoranda and directives has 
been provided on an ad hoc basis by 
medical advisors. This material assists in 
assessing the extent of disability for 
pension purposes and in performing 
pension medical examinations. Prior to 
pension reform, the table of disabilities 
was the responsibility of the Canadian 
Pension Commission. Since September 
1995, the table has been the Department’s 
responsibility. 


23.56 The table of disabilities provides 
descriptions of various medical conditions 
and diseases, which are used in 
determining the percentage of disability 
for pension purposes. It was designed 
primarily as an aid to physicians, making 
it difficult for adjudicators to use in 
making assessment decisions. For hearing 
loss claims, the Veterans Affairs Portfolio 
has developed guidance materials to 
determine the extent of disability for 
pension purposes. For other conditions, we 
found a lack of guidance material to link 


the medical condition claimed to the 
percentage of disability for pension 
purposes. Therefore, except for hearing 
loss claims, medical advisors are 
consulted in the assessment of claims. 
This practice highlights the need for 
additional guidance materials to support 
the table of disabilities. For example, in 
the case of an osteoarthritis hip condition, 
the table of disabilities provides only the 
following limited guidance: 


e mild — 10-25 percent; 
e moderate — 25-35 percent; and 
e severe — 35-50 percent. 


23.57 The table could be improved by 
expanding its scope and providing detailed 
criteria for assessing the extent of 
disability. These improvements would 
ensure greater consistency in making 
assessment decisions. 


23.58 Information on all pension claims 
is recorded in the Department’s Pension 
Status and Inquiry System. The 
Department’s Pension Disease 
Classification Manual provides a standard 
coding structure for coding medical 
conditions in this system. In Canada, this 
coding structure is unique to Veterans 
Affairs Canada. 


23.59 Interim assessments. Interim 
assessment decisions are rendered when, 
at the time of adjudication, there is 
sufficient information to establish 
entitlement to disability pension benefits 
but there is not sufficient information to 
assess the level of disability 
unconditionally. Interim assessments 
result in applicants receiving a pension 
award faster. 


23.60 Inthe sample of claims we 
reviewed, we noted that approximately 
one third of favourable first applications 
during the period July to December 1997 
were processed into payment using 
interim assessments. Based on 
departmental data, we calculated that it 
took an average of 3.5 months to schedule 
and complete a final assessment in cases 
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where an interim assessment had been 
made. 


23.61 We noted that the Department 
does not have a written rationale for the 
level of assessment determined for 
individual claims, such as the relevant 
information that was reviewed, the key 
factors analyzed or the degree of 
consideration given the various factors in 
rendering an assessment decision. Such 
documentation would facilitate review and 
would be useful in determining trends in 
decisions overturned on review and 
appeal. Decisions rendered by Veterans 
Affairs Canada are communicated to 
applicants in decision letters. These letters 
do not provide any explanation for the 
assessment of the level of disability. 


ee 
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23.65 As previously described, the 
Department’s officials make the decision 
on a first application for disability pension 
benefits. The results of a first decision are 
conveyed to applicants in a decision letter. 
If applicants are not satisfied with this 
first decision, they may notify the 
Department that they disagree. The 
Pension Act provides that the Department 
may, on its own motion, review a pension 
decision where it appears there was an 
error in a finding of fact or law. In 
addition, upon presentation of new 
evidence, clients may request a 
departmental review. 


23.66 ‘If applicants are not satisfied with 
the results of the first decision or 
departmental review, they may request a 


The Department needs 
to improve guidance 
material for making 


review by the Veterans Review and entitlement decisions 
: Appeal Board. After a review of the facts, 4 
me eas Mice cr ne a eg the Board may send a case back to the and assessing the 
in making entitlement decisions and in Department, may hold a review hearing or _ extent of disability. 
assessing the extent of disability for may in certain circumstances refuse to 
pension purposes. The guidance hear the case. 
materials should be revised periodically 23.67 _If a review hearing is scheduled, 
to reflect the trends of disability pension —_ applicants have the right to appear before 
claims. the Review Panel where they are usually 
represented by an advocate from the 
23.63 The Department should Bureau of Pension Advocates. The 
document the reasons for assessment advocate represents the applicant 
decisions and provide these reasons to throughout the review and appeal process 
applicants. without charge. 
Department’s response: The Department 23.68 If applicants are not satisfied 
agrees. Efforts are under way to address with the decision made by the Review 
these concerns through improvements to Panel, they may appeal the decision to an 
the Department's Table of Disabilities and Sppeal Panel of the Veterans Review and 
Medical Guidelines. These improvements Appeal Board. Only documentary 
should in turn enable the Department to evidence may be submitted at this stage, 
better articulate the reasons for and it is usually presented to the Appeal 
assessment adjudications for the benefit of | Panel by an advocate on behalf of the 
applicants and pensioners. applicant. 
23.69 Exhibit 23.10 shows the outcome 
Review and appeal process ofa sample of 126 first applications 
examined during our audit. Fifty-seven 
23.64 The claims adjudication process applications received favourable rulings at 
seeks to provide applicants with every the first decision level, nine received 
opportunity to establish their claims andto _ favourable rulings at the Veterans Review 
obtain disability pension benefits. The and Appeal Board review level and four 
review and appeal process allows each more received favourable rulings at the 
claim several adjudications. Board appeal level. At the first decision 
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Exhibit 23.10 


and Board review levels, a favourable 
ruling is defined as a ruling that is both 
favourable and acceptable to the 
applicant. 


23.70 In our sample, we found that 54 
percent of applicants who were 
dissatisfied with the rulings at the first 
level had the decision reviewed by the 
Board’s Review Panel. We also found that 


Decision Outcomes for Pension Application and Appeals Process 


126 first applications received 


83 percent of unfavourable or 
unsatisfactory Board Review Panel 
decisions went to an appeal hearing. As 
noted in the exhibit, these figures are 
conservative, as additional favourable 
decisions may be rendered with the 
passage of time. There is no time limit 
within which an applicant must request a 
review or an appeal of a pension decision. 


57 accepted 


(45% favourable ratio) 


First 
{ Decision 
vel 
69 unfavourable or Le 
unacceptable to client 
| 
: | Appeal Ratio = 54% 
32 withdrawn or no 37 claims appealed to 9 accepted 
further action VRAB! Review Panel (24% favourable ratio) 
VRAB 
2 Entitlement Reviews outstanding Review 
3 Assessment Reviews outstanding Level 
23 unfavourable or 
unacceptable to client 
| | Appeal Ratio = 83% 
4 withdrawn or no 19 claims appealed to VRAB! 4 accepted 
further action Appeal Panel (21% favourable ratio) VRAB 
Appeal 
| 9 Entitlement Appeals outstanding Level 


Notes: 


6 unfavourable 


| VRAB -— Veterans Review and Appeal Board 


4 


2 The 70 claims accepted is a conservative estimate of the eventual outcome because: 


70° of the initial 126 
(56%) were accepted 


e more requests for reviews or appeals may still be made (see claims under “withdrawn or no further action”); and 
e five review decisions and nine appeal decisions were outstanding at the end of our examination period (14 July 1998). 


Source: Office of the Auditor General, sample of 126 pension applications 
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Of the 37 claims appealed to the Board’s 
Review Panel, 13 claims, or 35 percent, 
were overturned at the review or appeal 
levels. 


23.71 During our review of Veterans 
Review and Appeal Board decisions and 
of analyses performed by the Department, 
we noted several reasons for decisions 
being overturned on review or appeal. 
Examples include the application of the 
“benefit of the doubt” provision of the 
Pension Act, the introduction of additional 
evidence (including oral testimony of the 
applicant at the review level) and different 
interpretations of the law, especially in 
matters of entitlement. In some cases, the 
reasons given by the Board for 
overturning departmental decisions were 
not fully articulated. At the Veterans 
Affairs Portfolio level, systematic analysis 
of decisions, based on clearly documented 
reasons for the decisions, could provide 
opportunities to improve the pension 
decision-making process. 


23.72 Inorder to improve the quality 
and efficiency of decision making, the 
Veterans Affairs Portfolio should carry 
out regular analyses of the reasons for 
decisions that are overturned by the 
Veterans Review and Appeal Board. 


Department’s response: The Department 
is committed to the provision of 
high-quality, consistent and timely 
decisions. We have previously conducted 
analyses of the reasons for decisions of the 
Veterans Review and Appeal Board. We 
will continue to do so on a regular basis in 
the future. 


Veterans Review and Appeal Board 
response: Agree. As has been the case 
since its creation, the Board continues its 
commitment to quality and the continuous 
production of fully articulated decisions. 
To achieve this goal, the Board will 
continue to provide intensive training to 
its members on all aspects of the 
adjudicative process, including the 
legislation, medical and legal issues, the 
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conduct of a hearing and decision writing. 
Quality assurance is a priority for Board 
staff and members. 


23.73 As described in paragraph 23.65, 
one of the legislative amendments 
effected under pension reform was Section 
82 of the Pension Act, dealing with 
departmental review. This review process, 
which can be used for certain cases such 
as an error in fact or the presentation of 
new evidence, is quicker and less costly 
than the Veterans Review and Appeal 
Board process. In addition, it does not 
require the applicant to forfeit any right to 
review or appeal a decision to the Board. 
The Department has made a commitment 
to render a decision on a departmental 
review within four weeks. The average 


turnaround time for reviews heard by the The departmental 
Veterans Review and Appeal Board is review mechanism is 
approximately four months. 

not being fully utilized. 


23.74 During our examination, we 
noted that for 3 of the 23 Board reviews 
and appeals we examined, additional 
evidence was submitted and formed the 
basis for overturning the previous 
decision. In addition, our review of 
departmental data shows that the average 
number of departmental reviews per 
month has fallen from 45 in 1997-98 to 
26 for the first five months of 1998-99. 


23.75 Toimprove the efficiency and 
timeliness of decision making in the 
disability pension program, Veterans 
Affairs Canada should examine 
opportunities to make greater use of the 
departmental review process. 


Department’s response: The Department 
is pursuing various means of improving 
the efficiency and timeliness of decision 
making for the disability pension program. 
It is agreed that in certain circumstances, 
use of the departmental review process 
could be exploited further. The 
Department will undertake to clarify and 
to communicate appropriate use of this 
tool. 
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A Veterans Affairs 
Canada-Canadian 
Forces co-ordination 
team has been 
established. 


The Changing Nature of the 
Department’s Clients 


23.76 Most Canadians would be 
interested to know that Veterans Affairs 
Canada continues to spend over $1 billion 
annually on veterans’ disability pensions. 
While pension expenditures have been 
relatively stable, the recipients of pensions 
have changed considerably. Today, the 
majority of recipients are either former or 
current peacetime members of the 
Canadians Forces or survivors 

(Exhibit 23.1). 


23.77 The Department has estimated 
that within 10 years, war service veterans 
will form the smallest group of disability 
pensioners. The largest client group of 
disability pension recipients will be 
survivors, followed by former or current 
peacetime members of the Canadian 
Forces. As shown in Exhibit 23.2, 
Canadian Forces disability pension clients 
are much younger than war service 
clients. As the pensioner population 
changes, it is important that the 
Department consider options for 
delivering services to clients efficiently 
and economically. 


23.78 Veterans Affairs Canada and 
National Defence have recognized the 
increasing prevalence of peacetime 
members of the Canadian Forces as clients 
of Veterans Affairs. The departments have 
been working together to enhance 
communication, to develop a closer 
relationship between the two departments 
and to improve Canadian Forces 
members’ access to Veterans Affairs 
Canada services and benefits. 


23.79 A Veterans Affairs-Canadian 
Forces co-ordination team has been 
established within Veterans Affairs 
Canada. This team was formed to provide 
a central co-ordination point in the 
Department for all matters concerning 
Veterans Affairs Canada and National 
Defence, to identify issues of mutual 
concern between both departments, to 


facilitate action and to monitor progress 
on improvement initiatives between the 
two departments. Each department has 
appointed a liaison officer to be part of an 
exchange program that will further 
enhance co-operative working 
arrangements. 


23.80 We reviewed Veterans Affairs 
Canada’s 1998-99 Report on Plans and 
Priorities, tabled as part of the Estimates. 
The document makes little mention of the 
Department’s role in relation to the 
peacetime members of the Canadian 
Forces or how the Department will change 
to fulfil this increasingly important role in 
the future. We expected that Veterans 
Affairs Canada would use this key 
accountability document to explain the 
changing nature of clients and the strategy 
for adapting to the changes. 


23.81 Inreporting to Parliament 
through its Report on Plans and 
Priorities, Veterans Affairs Canada 
should explain the role it plays in 
relation to the peacetime members of 
the Canadian Forces, the changes 
expected in its client base and its 
strategy for adapting to these changes. 


Department’s response: Veterans Affairs 
Canada is currently analyzing the 
medium- to long-term changes of its client 
base, including Canadian peacetime 
forces. The 1999-2000 Report on Plans 
and Priorities will report on the 
Department’s role concerning peacetime 
forces as well as on an approach to 
address these changes. 


Year 2000 Compliance 


Year 2000 compliance is a departmental 
priority 


23.82 The Year 2000 problem stems 
from the long-established practice of 
entering, storing, calculating and reporting 
dates in a six-digit format reflecting 
year/month/day. This practice began in the 
1950s and 1960s to save computer data 
storage space, which at the time was 
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limited and expensive. Many systems 
today still use six-digit date formats where 
the year is in a two-digit format. In such 
systems, the year 1998 is represented as 
“98” and the year 2000 as “00”. However, 
a two-digit year code of “00” may be 
interpreted by systems as the year 1900 or 
a “beginning of time” date, such as the 
date the system was implemented. Such 
erroneous interpretations of year 2000 
dates can lead to serious problems in 
systems that use dates for labelling, 
sorting, updating and other data 
manipulations involving dates. 


23.83 Many of the Department’s 
systems use a six-digit date format. Dates 
are used extensively for important 
calculations. For example, the effective 
date of a disability pension award is used 
to calculate the payment amount. An 
incorrect interpretation of a year 2000 
effective date could have a serious impact 
on payment calculations. A failure of any 
of the Department’s mission-critical 
systems could affect many Canadians 
since the systems provide a major source 
of income and access to services and 
benefits. 


23.84 In 1996, the Department initiated 
a Year 2000 project. The Treasury Board 
Secretariat has identified the Department’s 
health care systems and the pension and 
allowance systems as government-wide 
mission-critical systems. The Secretariat 
is closely monitoring progress toward 
Year 2000 compliance for all such 
systems. The Department is also tracking 
its Financial Management Information 
System as a mission-critical system 
because of its importance for 
administrative and financial control. We 
reviewed the Department’s actions to 
make the pension and allowance systems 
Year 2000-compliant. 


23.85 One of the departmental business 
renewal initiatives currently under way is 
the Benefits Redesign Project. As part of 
this initiative, a new Client Service 
Delivery Network system is being 
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implemented. This system is designed to 
be Year 2000-compliant. Since the 
original planned implementation date of 
1999 was too late for full resolution of 
Year 2000-related problems, the project 
was rescoped and rescheduled to be the 
primary Year 2000 system. In addition, the 
Department has adopted a strategy of 
converting existing systems as a 
contingency to ensure Year 2000 
compliance while implementing the new 
business renewal systems. 


Work on mission-critical systems is well 
under way 


23.86 The Department recognizes that 
the Year 2000 deadline cannot be changed 
and that project slippage could result in 
systems failure. Detailed progress on all 
project elements is regularly monitored 
and compared against the project 
schedule. As of July 1998, the project 
schedule and progress to date indicated 
that the Department’s government-wide 
mission-critical systems would be Year 
2000-compliant by December 1998. At 
the time of our audit, the Year 2000- 
compliant allowance system was 
operating and the pension system was 
being user tested. 


23.87 Implementation of the Client 
Service Delivery Network is planned to 
occur in phases, with the first release 
planned for December 1998 and the 
second release planned for March 1999. 
Through various initiatives, the 
Department has been able to secure 
adequate staff resources to carry out 
projects related to Year 2000 compliance. 
Project funding has also been secured. 
Total project costs, estimated as of July 
1998, are $51 million for the Veterans 
Affairs Portfolio’s systems. 


Dependency on non-departmental 
systems 


23.88 As is common with other 
organizations, the proper functioning of 
the Department’s systems requires 
exchanges of information with several 
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external systems. These interfaces expose 
the Department to risks beyond its control 
because it is dependent on external parties 
to perform certain tasks to ensure Year 
2000 compliance. The Department’s risk 
assessment indicates that it should be able 
to deliver services to clients even if these 
interfaces fail. At the end of our field 
work, testing of all but one of the 
interfaces had been scheduled. 
Negotiations were ongoing to ensure Year 
2000 compliance of the remaining 
interface. 


23.89 As of July 1998, the Department 
was beginning an assessment of its 
facilities and embedded systems to 
determine the extent to which they may be 
affected by the Year 2000 problem. 
Embedded systems consist of hardware 
and software that form a component of 
some larger system and are intended to 
operate without human intervention. 
These systems and facilities include, 
among others, telecommunications 
systems, building maintenance and energy 
management systems and security 
systems. 


23.90 The assessment was to be 
completed by the fall of 1998, with the 
majority of the required remedial 
expenditures to occur in 1999. The 
Department is working in conjunction 
with Public Works and Government 
Services Canada (PWGSC), Health 
Canada and other government 


departments on the compliance issues 
related to embedded systems. At the 
completion of our field work, the 
Department was surveying regional and 
district offices in co-ordination with 
PWGSC to develop an inventory of 
affected systems. 


23.91 The Department has developed 
an appropriate plan and is working toward 
resolving the Year 2000 problem in its 
government-wide mission-critical 
systems. We believe that the Department 
is appropriately managing the risks of the 
problem. However, because of embedded 
systems, facilities and the interaction of 
departmental systems with other 
government and non-government systems, 
it is impossible to give assurance that the 
Department will not suffer some effects 
from the Year 2000 problem. The 
Department is now beginning to address 
concerns about systems that are not 
mission-critical but could affect 
operations. 


Conclusion 


23.92 The Veterans Affairs Portfolio 
has been successful in meeting its 
commitment to significantly reduce 
turnaround times for processing pensions. 
It now needs to continue to enhance this 
process by taking the steps necessary to 
improve the consistency, efficiency and 
overall quality of disability pension 
decisions. 


23-24 


Report of the Auditor General of Canada — December 1998 


Veterans Affairs Canada 


y About the Audit 


Objective 


The objective of our audit was to determine whether the Veterans Affairs Portfolio (Veterans Affairs Canada 
and the Veterans Review and Appeal Board) is managing the disability pension program in a manner that 
ensures that high-quality disability pension decisions are made on a consistent basis and in a timely manner. 


Scope 


Our audit focussed on how the disability pension program is managed to achieve the legislated objectives of 
providing benefits to veterans and other eligible clients. We also examined the Department’s progress in 
making its pension and allowance systems Year 2000-compliant. 


Criteria 
We expected that: 


e Veterans Affairs Canada would have the means to ensure that high-quality disability pension decisions 
are made to consistently reflect the requirements and intent of governing legislation; 


e the Veterans Affairs Portfolio would collect and report accurate, relevant and reliable information on the 
length of time taken to reach decisions on disability pensions; and 


e the Veterans Affairs Portfolio would have made sufficient progress against its Year 2000 plan to ensure 
that concerns that could affect the timely processing of pensions have been addressed. 


Audit Team 


Assistant Auditor General: Don Young 
Principal: John O’Brien 
Director: Marilyn Rushton 


Donald MacNeill 
Heather McManaman 
Kimberlea Redden 


For information, please contact John O’Brien. 
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